Process multiple lines of unstructured logs and combine them as one structured record prior stashing them

Prologue

Logstash, in my opinion, is equipped with a toolset to process any kind of machine generated unstructured data into stash-able structured data. …


Overview

In modern application development web-requests are inevitable. For the security of those requests HTTPS is used and HTTPS depends on TLS…


Overview

Problem Statement

No internet connectivity at all for the servers and restricted internet connectivity for the workstations. We could only download files from the internet but this goes through a security scan before allowing us to download the file. …


As a beginner I thought the concept behind docker is similar to the concept behind Java, where you build once and deploy anywhere. For Java this capability is achieved through JVM (Java Virtual Machine). We need to install the corresponding compatible JVM for our Operating System. This JVM provides the isolation layer for Java programs to run on any platform (i.e on any OS). However, docker does not provide such isolation for docker containers. Windows based containers should run on Windows hosts, Linux based containers should run on Linux hosts.

Windows have introduced Hyper-V isolation to run Linux containers on…


Flask is a WSGI (Web Server Gateway Interface) compatible micro web framework written in Python. The easy learning curve and wide adoption have made Flask, one of the first choices for REST-API development in python. Due to the wide adoption, we can find a lot of tutorials and blog posts on ‘How to Deploy Flask Apps in Linux Servers’ in contrast to ‘Flask App Deployment in Windows’. However, there can be situations where one want to deploy one’s Flask applications on Windows-Servers too. In such a scenario someone might find this blog post handy and helpful.

Prerequisites


MySQL is one of the prominent open source RDBMSs. In the cases of traffic growth to DBMS and when in the cases of backing up data replication is there for the rescue. MySQL comes with a set of in-built replication mechanisms that can be utilized as the need arise. Mainly there are 2 replication setups, master-master replication, and master-slave replication. As the names imply master-master is used as a load balancing technique in the presence of high write load. In contrast, master-slave is utilized when read load is significantly high, and for data backup purposes. …


Debugging is inevitable when you are not a brilliant programmer. I’m a terrible programmer and debugging is the only way for survival:). Even though debugging flask applications with PyCharm IDE is as simple as bread and butter, tend to forget it time to time, thus thought of making note of it.

if __name__ == '__main__':
app.run()

Insert the above in any place you’d like. Make sure you have done the imports right. Here to run() method you can give whatever the parameters you desire.

Then, right-click on the script that you have put the above code snippet and click on debug. Add debug points wherever you want and happy debugging. Hola!

For people who know python and PyCharm, yep, you got it right :), that’s how you debug a python script with PyCharm.


Once in a while even for Linux/Ubuntu servers you might want remote desktop access. Yeah, you heard right! It is graphical user interface access to the operating system. It is true that we can do anything with Linux/Ubuntu operating systems with full privileged shell access. However, there can be some software applications that at some point of time might need actual GUI interactions to proceed. If you are at such a situation this blog post is for you.

Recently as per the instructions of team lead had to install ‘Wolfram Mathematica 11.3.0’ in one of our Ubuntu 16.04 instances at…


At the time of this writing I have been coding in python for five straight months. I came from Java/Javascript background and I was frustrated and always complaining about python packaging system. It was mainly because I did not understand the simple but effective concepts behind python packaging and also because I was thinking in terms of Java/Javascript. Now I vouch for python, python packaging, and pythonic way.

Without further ado, let’s get back to our topic. Python packaging totally based on the number of directories you have within a python project. …


Often python packages need to depend on files which are not python source (.py) files such as images, data tables, documentation, and etc. Those files need special treatment in order for setuptools to handle them correctly.

The mechanism that provides this is the MANIFEST.in file. This is relatively quite simple. MANIFEST.in is really just a list of relative file paths specifying files or globs to be included while packaging.

include README.rst
include docs/*.txt
include funniest/data.json

In order for these files to be copied at install time to the package’s folder inside site-packages, you’ll need to supply include_package_data=True to the setup()…

Thilina Madumal

Senior Software Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store