How can I monitor incoming http requests to my server without setting up a proxy?

Use tcpdump

• To monitor HTTP traffic including request and response headers and message body:

tcpdump -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

• To monitor HTTP traffic including request and response headers and message body from a particular source:

tcpdump -A -s 0 'src example.com and tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

• To monitor HTTP traffic including request and response headers and message body from local host to local host:

tcpdump -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' -i lo

• To only include HTTP requests, modify “tcp port 80” to “tcp dst port 80” in above commands
• Capture TCP packets from local host to local host

tcpdump -i lo

https://sites.google.com/site/jimmyxu101/testing/use-tcpdump-to-monitor-http-traffic

Also there is wireshark

I was able to manage this using logback-access library. Here are the steps:

• Download logback-0.9.30.zip from link text. This is the version compatible with our jboss 2.0.1 version. Please download compatible version

• Put logback-core-0.9.30.jar and logback-access-0.9.30.jar under /third-party/jboss/jboss-4.2.3.GA/server/<server-name>/lib/ folder.

• Insert <Valve className="ch.qos.logback.access.tomcat.LogbackValve"/> into third-party/jboss/jboss-4.2.3.GA/server/<server-name>/deploy/jboss-web.deployer/server.xml

• Create third-party/jboss/jboss-4.2.3.GA/server/esg_1.1/conf/logback-access.xml file with the below content

       <configuration>
        <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
      <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
            <!-- in the absence of a class attribute the <evaluator> element
                 defaults to ch.qos.logback.access.boolex.JaninoEventEvaluator -->
            <evaluator>
              <expression>event.getRequestURI().contains("appheartbeat.jsp")</expression>
            </evaluator>
            <onMismatch>NEUTRAL</onMismatch>
            <onMatch>DENY</onMatch>
          </filter>
      <encoder>
            <pattern>%fullRequest%n%n%fullResponse</pattern>
          </encoder>
        </appender>
  
        <appender-ref ref="STDOUT" />
      </configuration>
  

• If you decide to use filters, you will also need to add janino jar from this url link text under third-party/jboss/jboss-4.2.3.GA/server/<server-name>/lib/ folder.

• Add the filter and filter-mapping to third-party/jboss/jboss-4.2.3.GA/server/<server-name>/deploy/dvce.ear/dvce.war/WEB-INF/web.xml

   <filter>
     <filter-name>TeeFilter</filter-name>
     <filter-class>ch.qos.logback.access.servlet.TeeFilter</filter-class>
   </filter>
  
  <filter-mapping>
    <filter-name>TeeFilter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>
  

• Restart your server and watch the logs

For more info check links:

1. http://logback.qos.ch/recipes/captureHttp.html
2. http://logback.qos.ch/access.html

Important: If you look at the content of logback-access.xml file, you will see event.getRequestURI().contains("appheartbeat.jsp") line. Basically I am ignoring load balancer requests completely. You can use this filter and only monitor certain requests.

Here is a sample output

18:26:18,626  INFO (-0.0.0.0-8009-9) [                   STDOUT] GET /safari HTTP/1.1
host: esg-beta.onenetwork.com
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
connection: keep-alive
Accept-Encoding: gzip, deflate
user-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4
content-length: 0

HTTP/1.1 200 OK
Cache-Control: no-cache, must-revalidate, no-store
Set-Cookie: JSESSIONID=C5749E1E23D3D158B97AE42A04A084B7.plt106; Domain=.esg-beta.onenetwork.com; Path=/
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Length: 6792
Content-Language: en
Content-Type: text/html;charset=ISO-8859-1
18:26:18,781  INFO (-0.0.0.0-8009-9) [                   STDOUT] GET /oms/css/cc3/OpenSans.css HTTP/1.1
host: esg-beta.onenetwork.com
accept: text/css,*/*;q=0.1
connection: keep-alive
cookie: JSESSIONID=C5749E1E23D3D158B97AE42A04A084B7.plt106; BIGipServerESG-UAT-PLT=1778778890.20480.0000
user-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4
Accept-Language: en-us
referer:  https://esg-beta.onenetwork.com/safari 
Accept-Encoding: gzip, deflate
content-length: 0

HTTP/1.1 200 OK
ETag: W/"3021-1432849560000"
Last-Modified: Thu, 28 May 2015 21:46:00 GMT

@font-face {
  font-family: 'Open Sans';
  font-style: normal;
  font-weight: 300;
  src: local('Open Sans Light'), local('OpenSans-Light'), url( https://fonts.gstatic.com/s/opensans/v13/DXI1ORHCpsQm3Vp6mXoaTegdm0LZdjqr5-oayXSOefg.woff2)  format('woff2'), url( https://fonts.gstatic.com/s/opensans/v13/DXI1ORHCpsQm3Vp6mXoaTXhCUOGz7vYGh680lGh-uXM.woff)  format('woff');
}