Security Professional's Nightmare

Reportedly, NSA could install backdoors to Juniper devices. Oh my, oh my...

Spiegel reporter really has his fun, quoting:

"When it comes to modern firewalls for corporate computer networks, the world's second largest network equipment manufacturer doesn't skimp on praising its own work. According to Juniper Networks' online PR copy, the company's products are "ideal" for protecting large companies and computing centers from unwanted access from outside. They claim the performance of the company's special computers is "unmatched" and their firewalls are the "best-in-class." Despite these assurances, though, there is one attacker none of these products can fend off -- the United States' National Security Agency."

Do we want CPUGcon?

With Check Point User Group (CPUG), we are used to have not only a valuable place to exchange ideas and experience but also an annual event in Europe, the famous CPUG conference.

Starting from year 2008, CPUGcon was hosted in Chur, Switzerland and then in Munich, Germany. For many years it was, arguably, the most interesting technical event around Check Point security. Unfortunately, the interest to this conference seems to fade.

Year to year, we have less participants. This year in Munich we have reached all times low with only 50 attendees.

I am wondering what's going on. Is it the general interest to Check Point that is in decline? Is it something about administrative and organisational issues around the conference that drive people out? Is it something else?

Do we even want this conference to live on? Do we need it? And if yes, what can be done to get more people and more discussions there?

Three reasons why Check Point DDOS Protector is NOT a Check Point solution

Last year Check Point has added a number of so-called DDOS Protector appliances to its portfolio.

It is not a secret that this solution is in fact OEM of Radware DefensePro appliances, rebranded as Check Point.

There is no fun in stating obvious. It is not only about colours. Check Point's and Radware's security philosophies are essentially different. Here are three reason why I cannot consider box painted blue from the first picture to be a Check Point solution:

1. Security Architecture

In its firewalls Check Point performs inspection before forwarding the traffic. FW kernel is placed in between NICs and IP stack, so in general, traffic cannot be forwarded before a security decision to accept it is being reached*. 

In DefensePro/DDOS Protector solution network module and analysis module are separated. Traffic flows through the box while being analysed. Traffic is only interrupted once analysis module detects an attack. Detection is not immediate, and before anything, connectivity is maintained. 

2. Central management

In Check Point world management is a distinct entity. Although it can coexist with a firewall module in so-called standalone configuration, it is still a separate product. Working with management requires GUI tools. This set: GUI, management and enforcement point is called "three tier infrastructure". This separation helps building flexible centrally managed security systems.

With Radware DDOS solution, management is an integral part of the product in whole. It always is on the box, and to work with it, one uses WebUI instead of a standalone application. With such approach single box deployment is simple and straight forward while having a distributed centrally managed security system might be a challenge.

3. Integration

The only integration DDOS Protector has with Check Point is about logs. One cannot run it as part of centrally managed security system. DDOS Protector lacks built-in mechanism to share its own decision with the rest of the infrastructure and to reuse security decisions made by other parts of Check Point security system. Generally speaking, it has exactly the same level of integration as any other third party OPSEC product.

--------

*Strictly speaking, that is not exactly true for some specific cases of streaming-based inspection. Some IPS features require analysis of application stream and cannot be performed on per packet basis. The difference with Radware here is that packets are still going through FW kernel while being streamed, and thus are subject for FW inline stateful inspection anyway. FW needs not accept the flow through its security rulebase before any further in-depth inspection is performed.

Misleading certificate re-creation error

A customer of mine has to renew VPN certificate signed by VeriSign because it is about to expire.  The normal procedure would be to delete the existing one and to re-create is. The only issue with this is that it takes VeriSign a couple of days to process the request.

He has decided to create a new certificate with the same CA and to add it. Obviously, this does not work. The error is "certificate with the same DN already exists". The customer did not give up. He was trying and trying to use a different DN. The error was exactly the same.

In fact, it is the message which is not correct. According to SK61087, quoting:  

module can have only one certificate signed by a particular CA. Thus, when the new certificate is issued, you will be asked whether to replace any existing certificate signed by the same CA.

In this particular scenario certificate creation fails because there is already another certificate in place from the same CA. GUI error reports an issue which is completely different.

The only way is to delete the existing VeriSign certificate and to re-create it again. If there is a significant delays in the process, do not push policy on that FW till the renewal is fully done. 

Check Point: GO has to go

Some of you may have already seen Check Point announcement about GO solution, a.k.a. ABRA to reach end of sale on 31.12.2013.

I feel a bit blue about it. It is not only that GO was a unique virtualisation solution that never had anything close from the competitors. And not only about some customers of mine that bought these Check Point USB keys in bulks.

It is more about no alternative solution to replace it. Yet again, as sometime in the past with Interspect (I am genuinely surprised there are still some links to it on Check Point Web site), Check Point had a unique product idea but never managed to nurse it to maturity and wide recognition.

Sic transit Check Point GO, I guess...

Come to Munich to talk about Check Point

It is not too late to attend the most open and interesting technical discussion of the year - Check Point User Group conference in Munich. We have the best experts and we have beers, what could be a better combination?

Check the agenda, consider to stay two more days for custom training.

See you there, guys.

How to read some of Check Price licensing instructions (not a happy post)

Here is a logics exercise. Read the following quote from the Check Point licensing instruction:

"Starting from the 4800 model and above, each appliance running R75.40VS, R76 includes a total of 2 Virtual Systems (all SW Blades available on the GW are automatically supported on the free VS)"

Now, tell me, how many Virtual Systems can one run on a cluster of middle and high end appliances with the default, non VSX, licensing, according to Check Point?

Two, right?

Wrong. You can easily test it by yourself. Convert your physical cluster of R75.40VS or R76 to a VSX cluster. Once done, you will have your firewall converted to a Virtual System. That is VS number one. Now, try to add another VS. No, you cannot.

Why? Because there is only a single VS licensed, according to vsx stat:

Number of Virtual Systems allowed by license:     1

But where is the second one we should have? Where is our freebee FW?

Well, it was all just an illusion, according to a very recent Check Point SecureKnowledge case number 93415. Here is the quote from it (original orthography used): "the answer is that it comes with an initial gateway +1. so in the bottom line initial 2 vs license only covers VS0 and VS1."

Let me translate this for you from Checkpoint-ish. In plain English, that means you only have one VS licensed (VS1). VS0 is representing your physical cluster environment. After conversion to VSX it cannot route traffic anymore.

I wonder, how many customers have already misunderstood the quoted price list statement? R75.40VS is out for a year, and this confusion must be one year old. Then again, the mentioned SecureKnowledge case is only about two weeks old.

21.08.2013 - Update:

Peter Sandkuill, Check Point SE manager network security for Europe, was kind to reply to  this article. I am quoting his email:

"In the latest versions, starting R75.40vs, we consider VS0 to be the first virtual system. We can debate whether you want to use that exclusively for management (as a best practice) or deploy it as a full-fledged VS that runs just like other VS’s and happens to also accept management traffic as one of its interfaces is the management interface. If you convert a gateway all regular gateway interfaces become a member of VS0. This will route traffic just fine. Only if you decide to remove all interfaces and leave only a single one for management would it no longer route, as you would expect.
Especially when designing virtualization in smaller environments this is a compromise I have seen customers willing to make.

For the licensing part, VS0 is the licensed system. You get VS1 for free. Also note that when adding an additional VS package you lose that free VS. In example in a (to VSX) converted gateway you could have 2 * VS. VS0 and VS1. Adding a VS-10 package will give you a grand total of 11 * VS. VS0 and 10 additional ones."

Smart-1 upgrade to R75.40VS fails miserably with grub corruption and other issues

It is the second day on the row we are trying to upgrade two Smart-1 25 appliances from R75.10 to R75.40VS.

On the first trial we were doing SPLAT WebUI based upgrade. It has failed because of corruption of grab.conf that would not allow the machine to boot normally. Symptoms and solution are described in SK66029.

System was not bootable even after reverting to the original image of R75.10, so we have had to apply the solution anyway.

Hoping Gaia would be better, we have tried it now. Guess what? The machine is in a loop: booting and restoring image all the time.

grub.conf seems to be OK, but the system is no longer operational.

Hello, Check Point, any QA these days? We know the upgrade works on VMs, but what about testing your own alliance lines?

Personal invitation to Check Point Best Practices course

I will be teaching two days of Check Point Best Practices course in Munich as part of extended CPUG gathering.

I would like to use this opportunity to invite you to my class. We will be covering the following topics:

• Disaster recovery, backup techniques and tricks around them
• Upgrades and migrations done right
• Design of Check Point security systems
• Unknown and undocumented tools
• SPLAT and GAIA tricks

The course was originally started as a series of internal trainings for my colleagues. In the last several years it evolved into by far most popular training in my portfolio. Come and see why.

You can register to the course on CPUGcon registration page.

Thanks a lot for your interest.

Will I see you in Munich on CPUG conference?

Hello all!

Just a reminder, there is still some time to register to CPUGcon 2013. I would like to see you there during the conference and maybe even on one (better mine :-)) of the following classes.

Are you coming?

Check Point announces End of Sales for IP Appliances

Check Point will discontinue sales of the remaining IP appliances by the end of year 2013. That concludes the long and somewhat dramatic history of Nokia Security Appliances Business which was sold to Check Point almost 5 years ago.

Check Point customers may finally concentrate on a single appliance line, with GAIA as the main choice of OS.

With all its ups and downs, IPSO was quite interesting OS. For starters, it is based on BSD kernel while SPLAT and GAIA are based on RH Linux. In my personal opinion, GAIA still has some distance to cover before being as good as IPSO in terms of stability and features.

Anyhow, yet another page of network security history is about to be closed.

VSX provisioning bypass actually works on R75.40VS

In my previous post I have said VSX bypass debug commands do not work. I was wrong.
The trick works perfectly on R75.40VS Management, both SmartCenter and MDM.

Nevertheless, the weird part is that it only works if one types the commands manually and not by copy / paste. I guess it was only about some weird corruption when pasted from the buffer.

I am sorry for being wrong before and for all time spent on this topic by David Bar, Shahar Solomon, Eran Ashkenazi and other Check Point engineers. I appreciate the assistance.

VSX provisioning bypass trick does not seem to work on R75.40VS MGMT

WRONG, THIS WORKS. PLEASE SEE MY NEXT POST

I am doing lab trials for all kind of management and enforcement side upgrades for my customers, especially for ones usen VSX and MDM. As part of the MGMT sanity checklist, there are VSX provisioning checks to be sure there is no corruption of VSX objects and topology scripts.

Before R75.40VS there was a way to bypass actual connections to VSX clusters in case you are checking MGMT side only.

To do so, once would put the following set of debug commands in the Main CMA context on MDS machine:

fw debug fwm on TDERROR_ALL_VSXM_DBG_SKIP_PING=INFO 
fw debug fwm on TDERROR_ALL_VSXM_DBG_SKIP_INSTALL=INFO
fw debug fwm on TDERROR_ALL_VSXM_DBG_SKIP_PULL_SIC=INFO 


Once done, you could simulate topology changes for Virtual Systems to be sure scripts are properly handled. MGMT would generate a script without trying to connect to VSX cluster members and/or executing it on VSX cluster members.

Well, not anymore. With R75.40VS this trick is no longer working. It is even more interesting, just because it seems Check Point specialists are not aware of that. I have asked around and even opened a support call to get an answer.

I will keep you posted about the outcome.

R70 is out of support, VSX R65 is about to

I am not sure if you follow Check Point support lifetime details, but there are two things you need to know:

1. R70 versions are out of support from March 2013
2. VSX R65 will be out of support in May 2013

Take care and plan your upgrades as soon as you can. There is a hint for those who cannot do this in time: Check Point can extend support for half a year by request. Call you local contacts, it is not yet too late.

Removing traces of old versions - new scripts from Check Point

The most annoying disadvantage of upgrade in place is about having old traces in the file system. Basically, each upgrade leaves old product directories, so with every step you have less and less disk space available.

Removing old files manually is not exactly safe, so many prefer advanced upgrade.

The good news is it is no longer the best option. Check Point has two scripts, for MDM and for regular installations, that should clean your system after the upgrade.

Refer to SK91060 for regular systems and to SK65330 for MDM systems. According to the SK, the latter script is already integrated into R75.50 and R76 MDM installation packages.

With these tools one can upgrade in place and still have optimal disk space available quite close to a clean installation.

Finally, Check Point!

Could not push policy from R75.40VS to R67.10 VSX

A customer of mine could not install policy to a Virtual System on R76.10 VSX cluster after upgrading MDM servers to R75.40VS.

Policy verification was failing with multiple errors (some data removed):

INTERNAL ERROR in execval: optimization disabled: displacement too large
INTERNAL ERROR in execval: optimization disabled: displacement too large
ERROR: Table or domain are not allowed here
ERROR: table '<'quota_table'>' has no predefined format
ERROR: table '<'quota_table'>' has no predefined format
Compilation failed.
Operation ended with errors.

The messages look quite scary, but no worries.  The key here is quota_table.

In fact it is an old error from 2007, related to Network Quota being enabled in IPS profile. Disabling Network Quota fixes the issue. Check Point has SK32549 for that.

Although it is something known for long, I am wondering why it has been surfaced only after MGMT part being upgraded. 

Please let me know if you encountered this issue as well.

Check Point Security Report 2013 reveals scary picture

If you did not look into 2013 Security report done by Check Point, it is probably a good time to do so.
The document reveals quite disturbing figures: above 60% of networks are infected with bots. About the same amount of "protected" networks is open for P2P. More than half of organisations have DLP accidents. And so far, and so on.

Appliance sizing tool is available to partners

Remember we have talked about Check Point Appliance Sizing tool some time ago?

Finally, Check Point partners have it available through User Center under Quoting.

GAiA SNMP howto SK article

There is a fresh and very nice SecureKnowledge article about SNMP configuration on GAiA, with plenty of screenshots.

Thanks to Ben Peeri for writing it and letting us know.

User Center application for iPhone

If you are dealing with Check Point UserCenter on a daily basis, you might appreciate new Check Point iPhone app.

It allows you to search SecureKnowledge, access your support requests, see the latest security alerts and browse App Wiki.

Tufin Expert blog

I am starting Tufin blog here. Similar to CCMA blog, I will be using it as public working notes concerning various Tufin related topics.

VSX R67: Incorrect IP address of re-defined VLAN interface

I have got a support call the last week about inability to re-configure IP address on one of the Virtual Systems' VLAN interface on VSX R67.10 system.

To be more specific, say it was eth1.444 with IP address 192.168.xxx.yyy. During the migration to this VS, customer had to "hide" interface for some time. So he has changed its IP address to something like 1.1.1.1.

First time it has worked like a charm. But when he wanted to put the production IP address back, strange things began to happen. In the SmartDashboard it was all right, but instead of 192.168.xxx.yyy eth1.444 was still reported with 1.1.1.1 by ifconfig and cphaprob -a if commands.

Deletion and recreation fo the interface did not change the situation. Eventually the customer has opened a support call with us.

Apparently this is an known issue described in Check Point SecureKnowledge in sk67120.

The solution was to install policy on the VS after re-definition of the interface.

The issue is only specific to R67 and does not seem to appear with R65 and R75.40VS.

Israeli Ministry of Defence chooses Fortinet over Check Point

According to the report of Israeli Calcalist (Hebrew) Ministry of Defence in Israel is abandoning Check Point for Fortinet.

Considering Israeli Ministry of Defense was one of the first loyal customers of Check Point, this is a very unpleasant event for CP and a huge win for Fortinet. I hope upper management of Check Point taks this as a wakeup call, after series of similar situations with other customers around the globe moving away to other, rather less expensive, solutions.

Crossbeam passes Check Point in a BlueCoat

BlueCoat has announced the purchase of Crossbeam. The great drama of Crossbeam is now at its end.

It all started in 2011 when Check Point has announced 61000 appliance. Days of Crossbeam being a dominant of High End Check Point firewall vendor were obviously counted.

We all have heard stories and arguments about why Crossbeam should remain, about technology and features, about performance and advantages, but finally Crossbeam went to the market looking for a new partner.

As one of the Crossbeam certified experts, I admire their courage and spirit. I have liked the technology, and I have liked the people. I was very concern for them to go under and I am happy to see they have found a new business now.

Irony of this situation is that Crossbeam had suffered Check Point partnership twice.

First, there were C-series, the very first Check Point branded UTM appliances. They went extinct when Check Point had developed taste for its own UTM business. Crossbeam survived, with X-series. Not for long too long though.

I hope BlueCoat Will takes good care of them. I hope the people remain and the technology flourishes. 

All the good luck guys!