Dynamic Collection Settings
Categories:
3 minute read
Dynamic Collection Settings
Version 2.6.0 added a whole category of features named Dynamic Collection Settings. The general premise of each feature is that additional requests are made based on the target site’s observed state.
Collect Backups
Using --collect-backups means that for every file found during a scan, feroxbuster sends additional queries for the following default extensions:
~.bak.bak2.old.1.swp
Any url that wasn’t filtered out via status code, size, similarity, etc… is considered found.
Version 2.10.2 added the ability to specify a custom list of extensions to query:
feroxbuster -u https://some-example-site.com --collect-backups .bak .save .save~
Collect Extensions
Using --collect-extensions means that any found url that has an extension will be added as an extension to try for
all future requests.
There is a (large) default list of extensions that
are ignored by --collect-extensions. If --dont-collect is used, the default list isn’t used, and whatever values
are passed to --dont-collect are used instead.
Collect Words
Using --collect-words means that each valid response’s html is parsed, and words considered important are added
to the wordlist to be requested.
A word’s importance is determined via a Natural Language Processing model and is considered against the entire corpus of parsed html.
Examples
Collect Backups
Default extensions
feroxbuster -u https://some-example-site.com --collect-backups
Assuming a found file of:
200 GET 127l 292w 4161c https://some-example-site.com/index.php
The following additional requests are made to some-example-site.com:
index.php~(append extension)index.php.bakindex.php.bak2index.php.oldindex.php.1index.bak- (replace original extension).index.php.swp- (vim swap, prefix.and append.swp)
Custom extension list
feroxbuster -u https://some-example-site.com --collect-backups .save .save~
Assuming a found file of:
200 GET 127l 292w 4161c https://some-example-site.com/index.php
The following additional requests are made to some-example-site.com:
index.php.save(append extension)index.php.save~index.bak- (replace original extension).index.php.swp- (vim swap, prefix.and append.swp)
Collect Extensions
most useful with
--extract-links; may cause number of expected requests to explode
feroxbuster -u https://some-example-site.com --collect-extensions
Let’s say our wordlist contains five entries:
index.phpstuffthingsmostuffmothings
Assuming a found file of:
200 GET 127l 292w 4161c https://some-example-site.com/index.php
--collect-extensions would effectively add .php to feroxbuster’s internal state, as though .php were passed via
--extensions|-x. The result would be the following requests.
stuffstuff.phpthingsthings.phpmostuffmostuff.phpmothingsmothings.php
There are caveats to the requests that will be made, mostly due to timing of finding the extension vs. requests already in-flight, but conceptually, the above is true.
Collect Words
feroxbuster -u https://some-example-site.com --collect-words
Assume four pages were found:
https://some-example-site.com/doc1https://some-example-site.com/doc2https://some-example-site.com/doc3https://some-example-site.com/doc4
And their contents were as follows:
doc1-Air quality in the sunny island improved gradually throughout Wednesday.doc2-Air quality in Singapore on Wednesday continued to get worse as haze hit the island.doc3-The air quality in Singapore is monitored through a network of air monitoring stations located in different parts of the islanddoc4-The air quality in Singapore got worse on Wednesday.
The following endpoints would be requested, in addition to whatever was in the normal wordlist.
/gradually/network/hit/located/continued/island/worse/monitored/monitoring/haze/different/stations/sunny/singapore/improved/parts/wednesday