VISUALIZING WEB SERVER LOGS INSIGHTS WITH ELASTIC STACK– A CASE STUDY OF UMMAIL’S ACCESS LOGS
DOI:
https://doi.org/10.24191/mjoc.v3i1.4882Keywords:
Data Analytics And Visualization, Visualizing Insights, Infographics, Data Exploration, Elastic StackAbstract
One of the most significant information resources that often overlooked and it is mostly owned by the modern organization today is logs data. Likewise, logs data analytics is practised in many industries for different purposes, including website/system performance improvement, web development, information architecture, web-based campaigns/programs, network traffic monitoring, e-commerce optimization, marketing/advertising, etc. Many tools or approaches are available for this purpose, some are proprietary and some are open source. Studying the nature of these tools in finding the suitable and the right log analyzer in order to perform log analytics economically, efficiently and effectively will give advantages to the organization towards utilizing the primary source of information for identifying the system threats and problems that occur in the system at any time through Visualizing Insights of source using Elastic Stack. These kinds of threats and problems which existed in the system can be identified by analyzing the log file and finding the patterns for possible suspicious behaviour. A case study of UMMAIL’s access logs is proposed to visualise web server logs. The system administrator's concern can then be furnished with an appropriate infographics representation regarding these security threats and problems in the system, which are generated after the log files, are analysed. Based on this signs the administrator can take appropriate actions.
References
Agrawal, K., & Makwana, H. (2015). Data Analysis and Reporting using Different Log Management Tools. International Journal of Computer Science and Mobile Computing, 47(7), 224–229.
Al-Asadi, T. A., & Obaid, A. J. (2016). Discovering similar user navigation behavior in web log data. International Journal of Applied Engineering Research, 11(16), 8797–8805.
Badshah, G., Liew S. C., Zain, J.M., Mushtaq Ali (2016). Watermark Compression in Medical Image Watermarking Using Lempel-Ziv-Welch (LZW) Lossless Compression Technique. Journal of Digital Imaging, 29(2), 216-225. DOI 10.1007/s10278-015-9822-4
Bishop, I. D., Pettit, C. J., Sheth, F., & Sharma, S. (2013). Evaluation of data visualisation options for land-use policy and decision making in response to climate change.
Environment and Planning B: Planning and Design, 40(2), 213–233. https://doi.org/10.1068/b38159
Bodó, R., & Kouˇ, D. (2013). Efficient Management of System Logs using a Cloud. Brian Jackson. (2017). Top 10+ Log Analysis Tools - Making Data-Driven Decisions.
Retrieved November 9, 2017, from https://www.keycdn.com/blog/log-analysis-tools/ Čegan, L., & Filip, P. (2017). Webalyt : Open Web Analytics Platform.
Chainourov, B. (2017). Log Analysis Using Splunk Hadoop Connect. NAVAL POSTGRADUATE SCHOOL. Retrieved from https://calhoun.nps.edu/bitstream/handle/10945/55581/17Jun_Chainourov_Boulat.docx. pdf?sequence=1
Fuente, A. D. D., Andreassen, O. O., & Charrondière. (2015). Monitoring Mixed-Language Applications with Elasticstash, Logstash and Kibana ( ELK ). Proceedings of ICALEPCS2015, 786–789.
Goel, N., & Jha, C. K. (2013). Analyzing Users Behavior from Web Access Logs using Automated Log Analyzer Tool. International Journal of Computer Applications, 62(2), 975–8887. https://doi.org/10.5120/10054-4643
Hasani, Z., Jakimovski, B., Kon-popovska, M., & Velinov, G. (2015). Real Time Analytic of SQL Queries Based on Log Analytic, 78–87.
Iuhasz, G., Pop, D., & Drăgan, I. (2016). Architecture of a scalable platform for monitoring multiple big data frameworks. Scalable Computing, 17(4), 313–321. https://doi.org/10.12694/scpe.v17i4.1203
Jayakumar, V., & Alagarsamy, K. (2013). Analysing Server Log File using Web Log Expert in Web Data Mining. International Journal of Science, Environment and Technology, 2(5), 1008–1016.
Jayathilake, D. (2012). Towards structured log analysis. JCSSE 2012 - 9th International Joint Conference on Computer Science and Software Engineering, 259–264. https://doi.org/10.1109/JCSSE.2012.6261962
Keim Daniel, K. J., & Mansmann, G. rey E. and F. (2010). Mastering the Information Age Solving Problems with Visual Analytics. Mastering the Information Age Solving Problems with Visual Analytics, 57–86. https://doi.org/10.1016/j.procs.2011.12.035
Koga, I., & Almeida, E. S. (2016). File Transfer Log Analysis : A meteorological data center case study, 111, 1–13. https://doi.org/10.6062/jcis.2016.07.03.0111
Kumar, V., & Thakur, R. S. (2017). A Brief Investigation on Web Usage Mining Tools ( WUM ). ISSN, 2(1), 1–11. https://doi.org/10.21276/sjeat.2017.2.1.1
Liew, S.C. , Liew, S.W. and Zain, J.M., (2010). Reversible Medical Image Watermarking For Tamper Detection And Recovery With Run Length Encoding Compression, World Academy of Science, Engineering and Technology (WASET), Issue 72, December 2010,
pp. 799-803.
Lu, C., Zeng, H., Liu, J., Zhang, R., Chen, Y., & Yao, Y. (2017). Network Security Log Analysis System Based on ELK, (Cece), 554–559.
M. Fedorov, P. Adams, G. Brunton, B. Fishler, M.Flegel, K. Wilhelmsen, & R. Wilson. (2017). Leveraging Splunk for Control System Monitoring and Management. ICALEPCS 2017, 1–7. Retrieved from https://scholar.googleusercontent.com/scholar?q=cache:CbAw1NEck74J:scholar.google
.com/+logs+analytics+tools+&hl=en&as_sdt=0,5&as_ylo=2017&as_yhi=2017
Maria Minsker. (2017). Analytics Tools Are Still a Challenge for Many - eMarketer.
Retrieved December 24, 2017, from https://www.emarketer.com/Article/Analytics-Tools-Still-Challenge-Many/1015983
Mateik, D., Mital, R., Buonaiuto, N. L., Louie, M., Kief, C., & Aarestad, J. (2017). Using Big Data Technologies for Satellite Data Analytics. AIAA SPACE and Astronautics Forum and Exposition, 1–10. https://doi.org/10.2514/6.2017-5161
Mikula, A., Adamová, D., Adam, M., Chudoba, J., & Švec, J. (2016). Grid Site Monitoring
and Log Processing using ELK, 54–61.
Mitra, M. J. (2016). The Rise of Elastic Stack The Rise of Elastic Stack, (November). https://doi.org/10.13140/RG.2.2.17596.03203
Prakash, T., & Patel, K. (2016). Geo-Identification of Web Users through Logs using ELK Stack, 606–610.
Purnami, S. W., Zain, J.M., Tutut Heriawan, “An alternative algorithm for classification large categorical dataset: k-mode clustering reduced support vector machine”, International Journal of Database Theory and Application (IJDTA), Vol. 4, No. 1, March 2011, pp.
19-29
Shakti, M. K., & Garg, L. (2017). Web Log Analyzer Tools: A Comparative Study to Analyze User Behavior, 17–24. https://doi.org/978-1-5090-3519-9/17/$31.00@2017 IEEE
Smith, G. (2015). Log Analysis with the ELK Stack ( Elasticsearch, Logstash and Kibana).
Retrieved from https://www.linuxfestnorthwest.org/sites/default/files/slides/Log Analysis with the ELK Stack.pdf
Umarani, J., Silambarasi, A., & Thangaraju, G. (2016). Web Usage Mining Based Analysis of Web Site Using Web Log Expert Tool. International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE), 23(5), 33–36.
Valency Networks. (2016). Security Log Analyzer Tools | Pune Mumbai Hyderabad Delhi Bangalore India | Valency Networks. Retrieved October 17, 2017, from http://www.valencynetworks.com/articles/top-10-web-log-analyzers.html
Published
Issue
Section
License
Copyright (c) 2018 Harni Yusnidar Muhammad, Jasni Mohamad Zain
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
