LevelBlue Knowledge Base » Knowledgebase » Legacy Products » Secure Web Gateway » ICAP

Web Pages are Malformed on ICAP

Rudolf Kessler — Mon, 14 May 2012 08:06:23 GMT

Description
While using the ICAP topology, the browser shows malformed web pages.

Symptoms
On ICAP topology, web pages are being displayed with errors, pictures are not shown correctly and the whole page is malformed.

Cause
The REQMOD service is not properly configured on ICAP client side.

Solution
Make sure that REQMOD is properly configured on your ICAP client (Blue Coat, NetApp, ...).

This article applies to:

SWG 3000

SWG 5000

SWG 7000

This article was previously published as:

Finjan KB 1044

Bluecoat Proxy SG fails to "Sense Settings" in ICAP mode with version 10

Rudolf Kessler — Fri, 02 Mar 2012 07:08:55 GMT

This article applies to:

SWG 10.x
Bluecoat Proxy SGOS (seen with 5.5.x.x)

Symptoms:

Maximum number of ICAP connections is 16K (16,384) in SWG 10.x.
ICAP client service configuration fails to detect and apply this value in ICAP Response mode (sense Request Mode settings work fine). Value is unchanged, and remains under 16K.

Causes:

In SWG 10.x, the maximum number of possible established connections to the proxy or ICAP port is 16K, and can be divided between all available ICAP clients. The RESP/REQ mode ratio is 70:30 (hard-coded value), but the weighting of connections to each ICAP client can be configured as desired.
- Example: If using two ICAP clients weighted at 50:50, they are each capable of handling 8K (8,192) connections (see ICAP section under Scanner / "Devices"). Therefore, each ICAP client's theoretical maximum connections is 5,734 (70%) in RESP mode and 2,458 (30%) in REQ mode.
It appears that Bluecoat Proxy SG cannot use more than 4K (4,096) connections for one service. If SWG's "Sense Settings" tries to use a value higher than this, the configuration process fails to accept this higher value.

Resolution:

There are two options:

The preferred option is to change the weighting in SWG devices section and limit it to 35% per ICAP client (even if when combined they do not add up to 100%). Now "Sense Settings" should complete successfully.
Although not recommended, the second option is to skip "Sense Settings" and set 4096 manually in RESP mode.

Notes:

SWG's ICAP server responds correctly:

Bluecoat SGOS does not accept the value:

If set manually, Bluecoat SGOS returns an error:

Solution: Limit weighting to 35%, as this will return a valid number and "Sense Settings" will successfully complete.

ICAP Timeout Error When Browsing Through BlueCoat Proxy

Rudolf Kessler — Fri, 10 Dec 2010 09:07:11 GMT

Description
When browsing through a BlueCoat proxy with a Finjan scanner configured as an ICAP, client timeouts occurs.

Symptoms

The following error is shown when trying to browse to a site:

ICAP Error (icap_error)

An error occurred while performing an ICAP operation: Request timed out: Timed out while waiting for a response from the ICAP server.
There could be a network problem, the ICAP service may be misconfigured, or the ICAP server may have reported an error.

For assistance, contact your network support team.

Cause
This is a result of a misconfigured default connection timeout value in the BlueCoat (That was changed in SGOS ver 5).

Solution

To resolve this please increase the connection timeout for the RESP_MOD service as follows:

Software Version
all SWG versions, but not related to

This article applies to:

SWG 3000

SWG 5000

SWG 7000

This article was previously published as:

Finjan KB 1817

Preventing ICAP Scanning Loops

Rudolf Kessler — Mon, 09 Aug 2010 02:42:09 GMT

Description
In some environments, the only device that is allowed to connect directly to the web is the cache appliance. In such cases, the Vital Security Web Appliance must be configured to proxy its connections through the cache appliance. When the cache appliance is also an ICAP client, it is important to ensure that the Vital Security's transactions do not receive ICAP processing.

Symptoms
When transactions that originate from the Vital Security Web Appliance are scanned through ICAP, users might report lengthy scanning times for Java applets and ActiveX controls. If a sniffer is used to analyze the traffic, the administrator might see the same code (for example, a Java .class file) being requested multiple times. Additionally, updates might not install.

Cause
All Vital Security Web Appliances, regardless of role, must be able to connect to the Internet. The reason for this is broken out by role below:

All in One / Policy Server: These appliances must connect to the Internet in order to download both security and operating system updates.

All in One / Scanning Server: These appliances must connect to the Internet in order to prefetch content for scanning purposes. A common scenario in which prefetching occurs is when Vital Security analyzes a Java .class file that has dependencies on other Java .class files. Vital Security will request all of the associated .class files so that the entire applet can be scanned as a whole.

If transactions originating from the Vital Security appliances are passed back via ICAP, then the appliances may waste cycles scanning updates that do not need to be scanned. Additionally, prefetched content will be double-scanned, and this could result in scanning recursion and double-prefetching, which will increase system utilization and decrease overall performance.

Solution
The best way to prevent scanning loops is to configure the ICAP client (the cache appliance) so that transactions that originate from the IP addresses of the Vital Security Web Appliances do not receive ICAP processing. Neither REQMOD nor RESPMOD processing should be performed on transactions that originate from Vital Security. Additionally, if the IP addresses of the appliances are ever changed, the policy on the cache appliances should be updated so that transactions originating from the new IP addresses are not scanned.

Please note that these configurations are only necessary when the Vital Security Web appliances use an ICAP-enabled cache appliance as their upstream proxy. This includes any situations where the traffic might be transparently redirected to the ICAP-enabled cache appliance. However, when Vital Security appliances are allowed to connect directly to the Internet (without going through the cache appliance), there is no need to apply these configurations on the cache appliance.

Different cache appliances (and different versions of the same cache appliance) can vary in their configuration. For details on configuring a specific cache appliance to bypass ICAP for connections that originate from Vital Security's IP, we recommend contacting the vendor of the cache appliance. The vendor will be able to identify the best way to apply this configuration in your environment.

VSOS
all SWG versions

This article applies to:

NG 5000

SWG 3000 / NG 6000

SWG 7000 / NG 8000

This article was previously published as:

Finjan KB 1096

Pre-fetching requires internet access

Eric Hanson — Mon, 23 Mar 2009 00:00:00 GMT

Description
The prefetching mechanism is responsible for "grabbing" all relevant modules which are refered to as external modules from the internet. This is done with a clientless request (only if the modules are not yet present in the Fetcher).

Symptoms
A customer may encounter multiple issues if the scanners were not configured to have access to the internet.

ICAP errors (when the gateway is an ICAP client such as Blue Coat or Netcache).
Scanning latency
Multiple timeouts on browsing
Unscannable content.

Cause
As described above, part of the scanning process includes pre-fetching which requires direct access to the internet. If the scanners do not have direct access to the internet the prefetching mechanism will not be able to perform its tasks.

Solution

Enable scanner internet access on the network topology (e.g. Firewall allows the scanners to access the internet).
If there is no direct Internet access, in order to perform prefetching of Java classes for Applet scanning, ALL Scanning Servers must have the next proxy configured.
If you are using ICAP, ensure that the NG Appliance Scanning Server appears on the Access List.

Software Version
8.3.x
8.4.x

This article applies to:: NG 1000; NG 5000; NG 8000

This article was previously published as:: Finjan KB 1142

Why is ICAP client as cache proxy not caching content?

support finjan — Mon, 23 Mar 2009 00:00:00 GMT

Question
In ICAP topology, the ICAP client is also used as a caching proxy. However some of the content is not being cached. Why is this and how can it be solved?

Answer
In specific situations some of the content is not cached when using caching proxy and Vital Security Appliance as an ICAP server.
When using the Vital Security NG Appliance as an ICAP server with a caching proxy.

Any content request which gets the status page will not be cached on the caching proxy.

In order to have specific content cached on the caching proxy we should disable
the status page examination for this specific content.

In order to do so, please perform the following steps:
Log in to the Management Console.
Navigate to Settings -> Miscellaneous -> Status page.
Set the required content types to be cached under the ‘Don't Activate if Content Type includes following substrings’ text box.
Software Version
8.3.x
8.4.0

This article applies to:
NG 1000
NG 5000
NG 8000
This article was previously published as:
Finjan KB 1156

URL CGI parameters are malformed in ICAP with BlueCoat

support finjan — Mon, 23 Mar 2009 00:00:00 GMT

Description
Proxy chain with BlueCoat ICAP client and specific BlueCoat SGOS modifies URLs incorrectly.

Symptoms
When special characters are used as a part of search item and/or serve as part of url-cgi parameters, the resulting URLs may be wrong or malformed.

Cause
This behavior occurs due to SGOS recursively percent-encoding the percent sign (%) in percent-encoded characters that are embedded in the URL. In other words, the percent sign in percent-encoded characters is replaced with "%25". For example, "%2B" (percent-encoded plus sign), will become "%252B"

Lets review search query for "Finjan+Business" with the Google through this proxy chain:

The search results for this query are as following:

To find the modifications for URL parameters in this example review the network traffic:

Packet #1128
A client (10.194.150.155) issues a request to the BlueCoat (10.194.150.80) :
HTTP GET http://www.google.com/search?hl=en&q=Finjan%2BBusiness&btnG=Google+Search

Packet #1130
Blue Coat (10.194.150.80) modifies it and forwards a modified request to Finjan (10.194.150.117) :
GET http://www.google.com/search?hl=en&q=Finjan%252BBusiness&btnG=Google+Search

Packet #1133
Finjan (10.194.150.117) proceeds with the (wrong) request to the WebServer :
HTTP GET /search?hl=en&q=Finjan%2BBusiness&btnG=Google+Search

Solution
This behavior occurs when Blue Coat is running SGOS 4.2.4.x series.
Blue Coat technical support confirmed that this issue is scheduled to be resolved in SGOS 4.2.4.10 release.

In several cases solution for this behavior is to wrap such special characters with the spaces:

This search request is treated as expected :

And these are the results :

Software Version
N/A<

This article applies to:
NG 1000
NG 5000
NG 6000
NG 8000
This article was previously published as:
Finjan KB 1550