Indicator Management
Indicator Storage
Indicators sent to Blockade are hashed using the MD5[1] algorithm prior to being sent to the cloud service. This means that the true value of the indicator is unlikely to be known by any of the other analysts or end-users who receive them. We did this for two reasons:
- Sharing the exact indicator value is not always possible due to security concerns. By hashing the indicator, the true value is never directly shared, yet the meaning is still retained.
- These indicators are stored locally inside of the browser for every Blockade user. A large factor in hashing is that we can achieve a unique, fixed-length key that may describe something of great length. Hashing saves storage and obfuscates the true indicator values.
[1] MD5 is cryptographically insecure, but was chosen due to local client storage size limitations inside of localstorage. A smaller key size means that we can ultimately fit a significantly larger set of indicators into the client.
Behind the Scenes
Based on indicators sent in from analysts, Blockade will block traffic to those websites for end-users. It's important that analysts volunteering their time avoid making mistakes. The analyst toolbench takes indicators passed in and performs a number of processes on them before sending them out. Specifically, the following takes place:
- Indicators are cleaned of all bad characters
- Indicators are stripped down to just the core hostname value
- Indicators are de-duplicated
- Indicators are checked against hundreds of thousands of popular websites (Alexa and Cisco) using the base domain
- Indicators that were previously been sent and written to the cloud node are removed
Once the indicators are processed, requests are generated and queued up for sending. A time delay is placed between each submission of indicators to ensure that everything is working properly and to not hit the rate limit of the cloud node.