apache:logs:extract_all_user_agents_from_apache_logs
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
apache:logs:extract_all_user_agents_from_apache_logs [2023/07/17 12:11] – peter | apache:logs:extract_all_user_agents_from_apache_logs [2023/07/17 12:12] (current) – peter | ||
---|---|---|---|
Line 5: | Line 5: | ||
</ | </ | ||
- | (where “test.log” is the access logfile | + | * where “test.log” is the access logfile to analyze. |
Returns | Returns | ||
Line 19: | Line 19: | ||
</ | </ | ||
- | The first number (bolded) is the amount of times this spider/ | ||
<WRAP info> | <WRAP info> | ||
- | **NOTE: | + | **NOTE: |
+ | |||
+ | * Beware, these are not all crawlers, as the data is intermixed with actual human user traffic and other useful traffic. | ||
+ | |||
+ | |||
+ | In the example above, notice that the “Facebookexternalhit” user agent accessed the site 390,477 times per month. | ||
* That is roughly 541x per hour. Excessive!!!. | * That is roughly 541x per hour. Excessive!!!. | ||
* On the kill list, you go! | * On the kill list, you go! | ||
</ | </ |
apache/logs/extract_all_user_agents_from_apache_logs.1689595890.txt.gz · Last modified: 2023/07/17 12:11 by peter