R77.10 MDM clean installation fails in the lab

From time to time I am hearing complains about failing Multi Domain Managemet lab installations with R77.10.

Basically, after seemingly correct installation of MDM machine on VMware one cannot connect to MDM with Smart Domain Management tool. mdsstat command output is completely empty. mdsconfig command shows part of initial configuration wizard and then goes down with a segmentation fault. Management machine remains not configured, no matter what you do.

The reason for this is that it is not enough space in the root partition for MDS to be configured correctly. You have to configure at least 20 GB for your system partition to succeed with the installation.

Strangely enough, this is not mentioned in the Release Notes document. RN states 3.5 GB to be the minimal requirement for /root. That is true for any other installation but Multi Domain Management.

It is also not clear to me why this issue occurs in the first place, as MDM does not really use that much space anyway.

I hope Check Point fixes this soon, at least by amending the Release Notes. Before that, make sure your MDM virtual machine has enough space configured for the system partition. 

FW module is lost after reboot, analysis

Some of my colleague have experiences a strange failure on Gaia-based Check Point appliances lately.

On certain point, after reboot, FW module is not accessible and can only controlled via physical console connection. It comes up with some weird initial policy that does not allow HTTPS, SSL and/or SIC anymore. You can unload it with "fw unloadlocal" from console only. If one runs "fw stat" command, the message reports failure to connect to FW.

I have analysed the issue and found that it is related to the fact /etc/hosts file is missing the host entry for the FW.

The scenario is now clear for me. This only happens when you remove or disable an interface that was used to define MGMT IP address during the first time configuration wizard. Gaia is generating /etc/hosts automatically, and if management interface is removed or changed, hosts entry associated with the first NIC is also removed. After reboot OS cannot communicate with FW anymore, and the module connectivity shuts down completely.

To fix this, after re-defining management interface go to hosts configuration in WebUI and make sure the new management IP address is properly defined there with the module hostname. Same can be done from CLISH. Do not try to edit the file form bash with VI, this will not work.

I did not manage to find any SecureKnowledge entry for this scenario.

Notes about sync redundancy

During the last Advanced Check Point Troubleshooting course I have been asked about best practices to build sync redundancy with Gaia.

The question is not a simple as it sounds. The classic textbooks for ClusterXl recommend using two or more independent synchronisation interfaces marked as First and Second Sync. Although it was true for older versions, R7x changed the play.

Sk92804 "Sync Redundancy in ClusterXL" clearly states using multiple sync interfaces is obsolete. The new best practice is to build a bond interface defined as sync.

Now simple, you say? Not really. Using bond interfaces with Check Point is tricky. There are at least three SecureKnowledge articles that you should keep in mind, mostly for CP appliances:

• State of Sync interface configured on Bond interface is 'DOWN' for each Virtual System
  Solution ID: sk100450
• SecurePlatform / Gaia OS crashes on 12000 / 21000 appliance during configuration of Bond interface
  Solution ID: sk69442
• Incorrect count of Bond slaves in use after physical link down
  Solution ID: sk98160

Each one of them requires a fix. Only after three support fixes your sync should be fine.

Kernel debug flags revealed

As I have been advised yesterday night, Check Point has published another extremely interesting document: Kernel Debug Flags. It is a comprehensive list of all kernel modules (chains) of R77 and debug flags for them.

Anyone dealing with kernel troubleshooting may want to download this at once.

Sergei Shir, Check Point TAC engineer and SecureKnowledge content developer is responsible for this brilliant material. Thanks, Sergei, for writing this document and sharing it with the community.

Distribution note: Although the document is classified as "Restricted", it is available for anyone with a valid User Center account. Sergei has personally asked me to share this document with the community.

Forwarding Management logs from CMAs to CLMs

If you only log your GWs to CLMs and not to CMAs, it is not exactly convenient having Management audit logs still residing on CMAs.

sk27042 is addressing this matter, but it is grossly outdated. Here is a procedure to forward audit logs to CMAs that works for versions R75.40 and up:

• Make sure that the CMA is not specified as a Log Server for any Security gateway. If it is, these Security gateways should be reconfigured to redirect their logs to somewhere else (for instance to the CLM). 
• Use GUIDbEdit, connect to CMA in question, under "network objects" find . In the object properties, find log_server parameter and set the value to false. Then find use_loggers_and_masters parameter and change its property to true. Save DB and exit GUIDBEdit.
• Log in to CMA with smartDashboard and open CMA object, then go to Logs tab. 
• If the settings there are greyed out, change settings to control the log settings using SmartDashboard (press “here” link in the tab). Set up primary and secondary log location as required. 
• Install database on all MGMT objects.

Log into CLM with SmartTracker and check you now have Management logs coming in.

iPad Document security - follow up

Some people from Check Point are still sending me private messages concerning my post about document security iPad app. Some even try to make it personal. Some others are just interested in getting to the bottom of it.

To both groups: I did not make it up, and I have honestly described the issue. Some of you even have the detailed step by step explanation of my experience.

I am sorry if you are upset by that.

I could give a hand to second group to reproduce the issue. I am still not ready to do in on my personal iPad, because I am using it for business purposes, and I have to have it operational. Any other option will be carefully considered.

My door remains open for you.

Transparent Kerberos SSO with multiple GWs

Check Point Identity Awareness is a neat feature, especially with browser based transparent SSO authentication. It is also a challenging one. There is a lot of configurations to do on AD side, and that is not a strong domain for some of FW administrators. For example, I specifically started working with Check Point to be as far away as possible from a turmoil of Windows administration.

Jokes aside, there is something that Check Point documentation is not covering clear enough.

With Kerberos, one has to configure Kerberos Principal Name with a use account. Identity Awareness admin Guide is covering this point fairly well, on pages 58-59 (R77 version of the document). There is a caveat though. The document is written under assumption there is only a single FW or clsuter enforcing user identity with Kerberos. Ktpass command should how to map Kerberos parameters to the user account in the document are only working for a single portal URL.

What if one has more than one GW? Ktpass is no use here. Instead, administrators have to edit servicePrincipalName with Multi-valued String Editor to add multiple URLs there to enable IA working for the same user through multiple Identity Awareness enabled gateways. To simplify the config, just refer to this screenshot bellow.