Some of us are running kernel debug on Check Point FWs from time to time, especially students of my Advanced Check Point Troubleshooting Course.
There are many tricks and best practices, but one that I love the best is applying INSPECT traffic filter on the output. And guess what, this one is rarely used.
The issue is simple. When running FW kernel debug, we are trying to extract as much debug information as possible. Usually Check Point support engineers ask customers to run something like "fw ctl debug all". I assume they find filtering out gigs of text files very rewarding.
On my side, I am trying to keep things simple and controllable, hence enabling only necessary debug flag. Yet, by default you get all kernel output for all packets traversing your FW during the debug session, and that might be still too much when trying to debug a single faulty connection.
This is how "fw ctl debug" help output looks like in practice:
Usage:
fw ctl debug [-d] [-s ""] [-v (""|all)] [-k] [-x] [-m ] [-e expr |-i | -u] [+|-]
I have highlighted there the interesting part. If you ever used fw monitor, you know -e flag already. It stands for expression, a traffic filter based on INSPECT syntax.
Same deal here. You can actually limit quite a lot of unnecessary debug printout by filtering only the messages related to some particular traffic.
For example, adding
fw ctl debug -e "accept host(8.8.8.8);"
to your kernel debug commands you will get output related to traffic to and from the famous Google DNS server.
This saves lots of time and effort when analysing the debug info afterwards. As all the other parameters, this filter is reset with fw ctl debug 0.
Have fun.
There are many tricks and best practices, but one that I love the best is applying INSPECT traffic filter on the output. And guess what, this one is rarely used.
The issue is simple. When running FW kernel debug, we are trying to extract as much debug information as possible. Usually Check Point support engineers ask customers to run something like "fw ctl debug all". I assume they find filtering out gigs of text files very rewarding.
On my side, I am trying to keep things simple and controllable, hence enabling only necessary debug flag. Yet, by default you get all kernel output for all packets traversing your FW during the debug session, and that might be still too much when trying to debug a single faulty connection.
This is how "fw ctl debug" help output looks like in practice:
Usage:
fw ctl debug [-d
I have highlighted there the interesting part. If you ever used fw monitor, you know -e flag already. It stands for expression, a traffic filter based on INSPECT syntax.
Same deal here. You can actually limit quite a lot of unnecessary debug printout by filtering only the messages related to some particular traffic.
For example, adding
fw ctl debug -e "accept host(8.8.8.8);"
to your kernel debug commands you will get output related to traffic to and from the famous Google DNS server.
This saves lots of time and effort when analysing the debug info afterwards. As all the other parameters, this filter is reset with fw ctl debug 0.
Have fun.
Thanks for sharing the trick!
ReplyDeleteYou are welcome
ReplyDeleteThis was brought into the kernel in R71, however if the syntax was entered incorrectly it lead to the system hanging and requiring a reboot... so use it carefully. I haven't used it since tinkering in R71 I guess I should poke around with it and see how nice it plays now.
ReplyDeleteIt might have had issues with R71, but I am not aware of something bad after that. There is a known issue with VPN sessions without SecureXL, with R75-76. Nothing critical. Also, one can always test filter with fw monitor first
DeleteCool, good to know. I'll poke and test when I get these 4200's installed in my home lab. If I see anything odd I'll message you direct so you can make any additions to the article without a tonne of comments.
DeleteBy all means, do so :-)
Delete1) for FW Monitor syntax, please refer to sk30583: What is FW Monitor? (https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk30583)
ReplyDelete2) for Kernel Debug syntax, please refer to http://downloads.checkpoint.com/dc/download.htm?ID=41899
Thanks, Sergei, this is very useful. I will share both in a couple of separate posts.
DeleteNow, with p2., are there plans to include R80.10 GW debug modules into the doc? There are quite a few new modules there :-)
1) I will create the same PDF for R80.10 GW when it is officially released
Delete2) For Kernel Debug syntax, search for "kernel debug flags" in Support Center and go to the "Documentation" tab
3) For SecureXL Debug syntax, search for "securexl debug flags" in Support Center and go to the "Documentation" tab
Thanks again. Looking forward to see the updated version, especially for UP module :-)
DeleteI knew about this command 3 years ago, but its good you brought it up, as Im sure lots of clients never heard of it!
ReplyDeleteHi,
ReplyDeleteaccording to sk107078, there is need for a special fix for allowing to debug only specific connection.
So is this expression only for filtering but all flags for all connections are generated, which would mean it would not help on not putting load on the environment with the debug. Maybe only this is possible with sk107078 fix?
Regards
Norbert
Thanks, Norbert. The SK you mention is about extended functionality.
DeleteYou are correct about the debug load. However, if you use fw ct debug -e "..." syntax, only debug messages related to the filter will be put into the buffer.
Hi Valeri, I'm new to Checkpoint. Could you please advise me about the usage of various inspection points iIoO. Thanks in advance
ReplyDelete