PFSense - Squid - Options which affect the Cache

Cache Replacement Policy

Decides which objects will remain in cache and which objects are replaced to create space for the new objects.

Default: Heap LFUDA.

The cache replacement policies are:

Policy	Explanation	Comment
LRU	Squid's original list based LRU policy	Keeps recently referenced objects.
Heap GDSF	Greedy-Dual Size Frequency	Optimizes object hit rate by keeping smaller popular objects in cache so it has a better chance of getting a hit. It achieves a lower byte hit rate than LFUDA though since it evicts larger (possibly popular) objects.
Heap LFUDA	Least Frequently Used with Dynamic Aging	Keeps popular objects in cache regardless of their size and thus optimizes byte hit rate at the expense of hit rate since one large, popular object will prevent many smaller, slightly less popular objects from being cached. If using the LFUDA replacement policy you should increase the value of maximum_object_size above its default of 4096 KB to maximize the potential byte hit rate improvement of LFUDA.
Heap LRU	LRU policy implemented using a heap

The policies utilize a dynamic aging mechanism that prevents cache pollution that can otherwise occur with frequency-based replacement policies.

See https://www.hpl.hp.com/techreports/1999/HPL-1999-69.html and https://fog.hpl.external.hp.com/techreports/98/HPL-98-173.html.

Low-Water Mark in % (cache_swap_low)

The low-water mark for AUFS/UFS/diskd cache object eviction by the cache_replacement_policy algorithm.

Default: 90.

This defines when the replacement should take place.

Replacement begins when the swap (disk) usage is above the low-water mark and attempts to maintain utilization near the low-water mark.

As swap utilization gets close to high-water mark object eviction becomes more aggressive.

If utilization is close to the low-water mark less replacement is done each time.

WARNING: If you have a large cache, 5% could be hundreds of MB. If this is the case you may wish to set this number closer together.

High-Water Mark in % (cache_swap_high)

The high-water mark for AUFS/UFS/diskd cache object eviction by the cache_replacement_policy algorithm.

Default: 95.

This defines when the replacement should take place.

Replacement begins when the swap (disk) usage is above the low-water mark and attempts to maintain utilization near the low-water mark.

As swap utilization gets close to high-water mark object eviction becomes more aggressive.

If utilization is close to the low-water mark less replacement is done each time.

WARNING: If you have a large cache, 5% could be hundreds of MB. If this is the case you may wish to set these numbers closer together.

Hard Disk Cache Size

Amount of disk space (in megabytes) to use for cached objects.

Set to 50% of the hard disk size.

Hard Disk Cache System (cache_dir)

This specifies the kind of storage system to use.

Storage System	Description	Comment
usf	The old well-known Squid storage format that has always been there.
aufs	Uses POSIX threads to avoid blocking the main Squid process on disk I/O. (Formerly known as async-io.)
diskd	Uses a separate process to avoid blocking the main Squid process on disk I/O.
null	Does not use any storage. Ideal for embedded.

Level 1 Directories

Specifies the number of Level 1 directories for the hard disk cache.

Default: 16.
My setting: 64.

Each level-1 directory contains 256 subdirectories, so a value of 256 level-1 directories will use a total of 65536 directories for the hard disk cache.

This will significantly slow down the startup process of the proxy service, but can speed up the caching under certain conditions.

Minimum Object Size

Objects smaller than the size specified (in kilobytes) will not be saved on disk.

Default: 0 (meaning there is no minimum).

Maximum Object Size

Objects larger than the size specified (in megabytes) will not be saved on disk.

Default: 4 (MB).
My setting: 1024.

Objects larger than this size will NOT be saved on disk.

The value is specified in kilobytes.

If you wish to get a high BYTES hit ratio, you should probably increase this (one 32 MB object hit counts for 3200 10KB hits).

Leave this value low if you wish to increase the speed more than what you want to save bandwidth.

NOTE: If using the LFUDA replacement policy you should increase this value to maximize the byte hit rate improvement of LFUDA!

See replacement_policy section for a discussion of this policy.

Memory Cache Size (cache_mem)

Specifies the ideal amount of physical RAM (in megabytes) to be used for In-Transit objects, Hot Objects and Negative-Cached objects.

Default is : 64MB.
My setting: 2048.

Data for these objects are stored in 4 KB blocks. This parameter specifies the ideal upper limit on the total size of 4 KB blocks allocated.

In-transit objects have priority over the others. When additional space is needed for incoming data, Negative-cached and Hot objects will be released. In other words, the negative-cached and hot objects will fill up any unused space not needed for In-transit objects.

If circumstances require, this limit will be exceeded. Specifically, if your incoming request rate requires more than cache_mem of memory to hold In-transit objects, Squid will exceed this limit to satisfy the new requests. When the load decreases, blocks will be freed until the high-water mark is reached. Thereafter, blocks will be used to store hot objects.

In-transit objects or current requests	These are the objects related to the current requests and they have the highest priority to be kept in the cache space in RAM. These objects must be kept in RAM and if there is a situation where the incoming request rate is quite high and we are about to overflow the cache space in RAM, Squid will try to keep the served part (the part which has already been sent to the client) on the disk to create free space in RAM.
Hot or popular objects	These objects or web documents are popular and are requested quite frequently compared to others. These are stored in the cache space left after storing the in-transit objects as these have a lower priority than in-transit objects. These objects are generally pushed to disk when there is a need to generate more in RAM cache space for storing the in-transit objects.
Negatively cached objects	Negatively cached objects are error messages which Squid has encountered while fetching a page or web document on behalf of a client. For example, if a request to a web page has resulted in a HTTP error 404 (page not found), and Squid receives a subsequent request for the same web page, then Squid will check if the response is still fresh and will return a reply from the cache itself. If there is a request for the same page after the negatively cached object corresponding to that page has expired, Squid will check again if the page is available. Negatively cached objects have the same priority as hot or popular objects and they can be pushed to disk at any time in favor of in-transit objects.

WARNING: This tag does not specify the maximum process size.

It places a limit on one aspect of squid's memory usage.

Squid uses memory for other things as well.

Process will probably become twice or three times bigger than the value put here.

While deciding the RAM space for caching, we should be neither greedy nor paranoid. If we specify a large percentage of RAM for caching, the overall system performance will suffer as the system will start swapping processes in case there is no free RAM left for other processes. If we use a very low percentage of RAM for caching, then we’ll not be able to take full advantage of Squid’s caching mechanism.

Maximum Object Size in RAM

Objects greater than this size (in kilobytes) will not be attempted to kept in the memory cache.

Default: 256 (KB).
My setting: 1024.

This should be set high enough to keep objects accessed frequently in memory to improve performance at the same time low enough to keep larger objects from hoarding cache_mem.

Memory Replacement Policy

The memory replacement policy determines which objects are purged from memory when space is needed.

Default: Heap GDSF.
My setting: Heap LFUDA.

The cache replacement policies are same as cache_replacemen_policy:

Policy	Explanation	Comment
LRU	Squid's original list based LRU policy	Keeps recently referenced objects.
Heap GDSF	Greedy-Dual Size Frequency	Optimizes object hit rate by keeping smaller popular objects in cache so it has a better chance of getting a hit. It achieves a lower byte hit rate than LFUDA though since it evicts larger (possibly popular) objects.
Heap LFUDA	Least Frequently Used with Dynamic Aging	Keeps popular objects in cache regardless of their size and thus optimizes byte hit rate at the expense of hit rate since one large, popular object will prevent many smaller, slightly less popular objects from being cached. If using the LFUDA replacement policy you should increase the value of maximum_object_size above its default of 4096 KB to maximize the potential byte hit rate improvement of LFUDA.
Heap LRU	LRU policy implemented using a heap