Anomalous Behaviour and Threat Detection on Mobile Web in Safe Environment
The requirement was to make the internet activity secured from threats & unusual behaviour. There should be a proxy based server solution, proxies were used to protect the privacy of the communication that takes place between two users.
The server was expected to perform actions like penetration testing, site vulnerability analysis, anomaly detection etc. Upon the analysis performed by the server, users were to be notified with the severity of the website status. User gets an option to blacklist or whitelist a particular website, so that they could decide on whether or not to visit the particular website in the future.
- Retrieving web traffic from non-rooted phones.
- Time consumed to get the sites scanned and penetration testing were higher.
- The availability of network logs.
- A huge data storage is required, as there are a lot of data to be saved.
- Retrieval of web traffic in a non-rooted phone, was tackled by making custom proxy library along with credential storage certificate to properly track web traffic results.
- The challenge of time consumption in the penetration technique was reduced by the polling and queuing methods.
- The accuracy of site scanners were made effective by the installation of multiple vulnerability penetration scanners.
- Trained test phase data to identify behaviors of commonly visited websites. The training was then scheduled as a CRON job for the frequent update for threat signature model.
- Time series database backed by HBase were used to store huge amount of data.
- The solution provided a proxy based server, that is now capable of site monitoring and threat notification.
- Universal blacklist is maintained in order to block the sites having malicious content.
- A personalised blacklist can be created by the user, which could further help in parental controls.
- Java 8
- OWASP ZAP/ Subgraph Vega