Difference between online and no-online log fields

[syunusic]

When I use ONLINE_MODE="yes" this is what I get in most of the log lines:
time=2016-06-02 00:59:17|action=accept|orig=192.168.100.66|i/f_dir=inbound|i/f_name=eth3|has_accounting=1|uuid=<574faf15,00000021,4264a8c0,c0000000>|product=VPN-1 & FireWall-1|rule=44|rule_uid={E9C085F7-9DE8-4893-B1AF-86D9D255537F}|rule_name=Salida Google Apps, Youtube|service_id=UDP-443|src=192.168.226.51|s_port=65193|dst=64.233.186.100|service=443|proto=udp|xlatesrc=131.0.54.4|xlatesport=52436|xlatedport=0|NAT_rulenum=internal|NAT_addtnl_rulenum=internal|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={6D5AC528-0F96-9540-86C2-C3AE325686BA};mgmt=salvatore1;date=1464787912;policy_name=Conf2014]|origin_sic_name=CN=Firewall1000,O=salvatore.domain.com.q4a2j9

When I use ONLINE_MODE="no", this is what I get from the SAME log line (watch the UUID):
time=2016-06-02 00:59:17|action=accept|orig=192.168.100.66|i/f_dir=inbound|i/f_name=eth3|has_accounting=1|uuid=<574faf15,00000021,4264a8c0,c0000000>|product=VPN-1 & FireWall-1|rule=44|rule_uid={E9C085F7-9DE8-4893-B1AF-86D9D255537F}|rule_name=Salida Google Apps, Youtube|service_id=UDP-443|src=192.168.226.51|s_port=65193|dst=64.233.186.100|service=443|proto=udp|xlatesrc=131.0.54.4|xlatesport=52436|xlatedport=0|NAT_rulenum=internal|NAT_addtnl_rulenum=internal|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={6D5AC528-0F96-9540-86C2-C3AE325686BA};mgmt=salvatore1;date=1464787912;policy_name=Conf2014]|origin_sic_name=CN=Firewall1000,O=salvatore.domain.com.q4a2j9|start_time= 2Jun2016 0:59:17|segment_time= 2Jun2016 0:59:17|elapsed=0:00:00|packets=18|bytes=8002|client_inbound_packets=9|client_outbound_packets=9|server_inbound_packets=9|server_outbound_packets=9|client_inbound_bytes=2495|client_outbound_bytes=5507|server_inbound_bytes=5507|server_outbound_bytes=2495|client_inbound_interface=eth3|server_outbound_interface=eth2|__pos=7|__nsons=0|__p_dport=0

As you see, everything is the same until "origin_sic_name=CN=Firewall1000,O=salvatore.domain.com.q4a2j9". Why is this difference? Another thing is that there are some log lines that have this extra fields in both modes, but their values are totaly different. In the one with ONLINE_MODE="yes", the values are much smaller:
ONLINE_MODE="yes":
time=2016-06-02 04:28:26|action=accept|orig=192.168.100.66|i/f_dir=inbound|i/f_name=eth3|has_accounting=1|uuid=<574fe01a,00000016,4264a8c0,c0000000>|product=VPN-1 & FireWall-1|rule=44|rule_uid={E9C085F7-9DE8-4893-B1AF-86D9D255537F}|rule_name=Salida Google Apps, Youtube|service_id=UDP-443|src=192.168.240.75|s_port=63368|dst=209.85.239.142|service=443|proto=udp|xlatesrc=131.0.54.4|xlatesport=31249|xlatedport=0|NAT_rulenum=internal|NAT_addtnl_rulenum=internal|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={6D5AC528-0F96-9540-86C2-C3AE325686BA};mgmt=salvatore1;date=1464787912;policy_name=Conf2014]|origin_sic_name=CN=Firewall1000,O=salvatore.domain.com.q4a2j9|start_time= 2Jun2016 4:28:26|segment_time= 2Jun2016 4:28:26|elapsed=0:00:00|packets=5|bytes=1602|client_inbound_packets=5|client_outbound_packets=0|server_inbound_packets=1|server_outbound_packets=5|client_inbound_bytes=1602|client_outbound_bytes=0|server_inbound_bytes=30|server_outbound_bytes=1604|client_inbound_interface=eth3|server_inbound_interface=eth2|server_outbound_interface=Mgmt|__pos=2|__nsons=0|__p_dport=0

ONLINE_MODE="no":
time=2016-06-02 04:28:26|action=accept|orig=192.168.100.66|i/f_dir=inbound|i/f_name=eth3|has_accounting=1|uuid=<574fe01a,00000016,4264a8c0,c0000000>|product=VPN-1 & FireWall-1|rule=44|rule_uid={E9C085F7-9DE8-4893-B1AF-86D9D255537F}|rule_name=Salida Google Apps, Youtube|service_id=UDP-443|src=192.168.240.75|s_port=63368|dst=209.85.239.142|service=443|proto=udp|xlatesrc=131.0.54.4|xlatesport=31249|xlatedport=0|NAT_rulenum=internal|NAT_addtnl_rulenum=internal|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={6D5AC528-0F96-9540-86C2-C3AE325686BA};mgmt=salvatore1;date=1464787912;policy_name=Conf2014]|origin_sic_name=CN=Firewall2000,O=salvatore.domain.com.q4a2j9|start_time= 2Jun2016 4:28:26|segment_time= 2Jun2016 4:28:26|elapsed=0:00:01|packets=4125|bytes=3803464|client_inbound_packets=1370|client_outbound_packets=2753|server_inbound_packets=2757|server_outbound_packets=1374|client_inbound_bytes=109950|client_outbound_bytes=3693428|server_inbound_bytes=3693548|server_outbound_bytes=110182|__pos=7|__nsons=0|__p_dport=0

Any clues on this?

[syunusic]

I did more research and I'm possitive the problem is with the ONLINE_MODE="yes" flag. When the log has has_accounting=1, only has the fileds related to bytes and amount of packets if ONLINE_MODE is set to "no". The problem is I need to process logs in realtime. Please help!

[adepasquale]

FW1-LogGrabber retrieves all available fields by default.

My guess is that fields like the number of bytes/packets sent/received are not known immediately, but only when the connection is terminated. This might be the reason why some fields are not present when using ONLINE_MODE="yes", or have a smaller value.

[syunusic]

I thought the same, but If I take the SAME log line, first as online and then as not online, the second one has the account ones. It's weird, but is very consistent. Almost 99,9% of the time happens this way. This leads me to a question: nobody has online reporting with # of bytes or # of packets? I think that's the most important part of having firewall log s processed.

[adepasquale]

I have only used online mode without considering the accounting part, so I'm sorry but I can't be of any help. Have you considered asking Checkpoint directly?