Why I Love Software Engineering: The Three Body Answer
“What do you want to be when you grow up?”
“(Shouting) A Pilot!!!!”
I have been going through a slump lately, job wise. The pandemic has been exceptionally tough on everyone, blurring the line between personal and work lives. As the old saying goes, “too much of everything is bad”, I think the pandemic-borne extra long work hours have taken their toll on our work ethics. I believe most of us have reached new milestones and created personal records when it comes to over working themselves. I wonder how many people can assuringly and truthfully say that they love their jobs in these times. No doubt, complaining about your job is a privilege on its own as many have lost their only source of income. Shout-out to everyone who have toiled countless hours to traverse the covid landscape.
BUT does that mean I now hate what I actually do in my job? HELL NO! I love it!
Why do I love being a software engineer?
The short answer is: because I get to create and watch my creation solve real world problems. Isn’t that the best form of gratification? To watch something that you build make a real impact? In one way it’s kind of a parent-child endearment. Parents feel ecstatic when their children do good, they feel proud when the kids handle tough situations and don’t give up. I feel the same kind of sentiment when I see a system I have built run for years, handling rogue requests, over enthusiastic users and changing landscape.
The long answer is a 3-body answer:
It keeps your brain on its toes
Software engineering doesn’t let you slack off, it keeps throwing new challenges to your face which keeps the intellectual inside you alive. In most of our tasks, we get a situation and a desired outcome. Then, it’s upto the engineers to use a combination of existing and new tools at their disposal and build an entire infrastructure that does a hell lot of things, besides just giving a desired output. Getting the final results is easy, but building a full-proof system that can handle the idiosyncrasies or humans and machines and still keep running is the real deal.
Let me explain with a small example of a problem for which I had to build a solution. Suppose your application handles personal data of your users, maybe their emails, phone numbers, credit card info etc. Such data is called PII: Personally Identifiable Information. Companies usually have checks in place so that such PII data is kept out of hands of tech teams. But there maybe scenarios where an internal business team needs to access this data for their day to day work through your application. Now what if one frustrated employee decides to do some damage? He decides to download some user data and make it public! That could be very devastating for the users and the reputation of the company. Now, coming to the real ask, in simple terms, we need to build a system that tracks and logs every access to that PII data through your application, especially data download.
The answer seems simple, right? Log The Data Access Action! Logging data is THE most common and most important part of software building. It helps you track the application flow and is an indispensable tool to debug problems. But just logging the action into a file won’t help you, how would you search a particular user accessing a particular piece of information? You need to model your data in such a way that it is well structured to handle queries. But we won’t talk about that, else this blog will become a full system design solution. We’ll just limit ourselves to tracking the user action and forming the data packet to be sent to an external system. Let’s just assume that we submit a data packet to a streaming platform like Kafka, which then handles formatting the data and storing it in a queryable form.
Now how do you track the user’s action of accessing PII data? Let’s plugin a logic that, whenever the user loads up the app or clicks a button to see the user info, or updates the PII data of a user or downloads this data, does one simple thing: gets the user, gets the current time, gets an identifier to the actual data that was accessed, gets the IP address of the system that the user was using, and sends it over to Kafka. These data points are enough to pin point the user who tried to access the PII data. Simple, right? The work is done!
NO! In fact, it’s just the beginning. What’s the true merit of this system? That it tracks user actions in certain scenarios. What’s that one thing we cannot tolerate in the system? Missing a user’s access log getting captured.
As I said before, getting the result is easy, building a robust system is hard. Let’s now think about where things can go wrong. What if your application crashes just after downloading the data but before sending the LOG to kafka? What if there is a network issue and the LOG never reaches Kafka? What if kafka itself is down? What if the LOG building is complex and it starts interfering with the actual application’s logic? What if the request from the user fails, and he is unable to see the data? Would you still track it?
You see the layers now? It’s like peeling an onion, each layer poses a new problem. And when you a build a software application, you need to engineer it in such a way that it can handles failures and is intelligent enough to gracefully recover from most error scenarios on its own, without human intervention.
I’m sure that there are other jobs which provide you the same gratification and same intellectual challenges. But what makes software engineering unique is the fact that it’s all intangible! Every tool that you are ever gonna need to is available on your small laptop! You code, run, break, fix, improve, destroy, collaborate all on your small plastic computer. You can literally sit in a cafe and push a feature that can change how people all over the world interact!
I find that cool!
The curious case of bugs in computers
It’s not known as to who first coined the term “bug” for mistakes, but there are several stories around it. One such story which I find interesting is:
In 1946, when Grace Hopper was released from active duty, she joined the Harvard Faculty at the Computation Laboratory where she continued her work on the Mark II and Mark III. Operators traced an error in the Mark II to a moth trapped in a relay, coining the term bug. This bug was carefully removed and taped to the log book. Stemming from the first bug, today we call errors or glitches in a program a bug.
Bugs are another interesting peculiarity that stem out of software engineering. It has several connotations attached to it, in simple terms, whenever a software system doesn’t behave as expected, it’s a bug.
As you grow in your career as a software engineer and you gain experience, the bugs you introduce reduce in frequency but grow in impact and complexity.
I love catching and solving bugs!
This is the second concept that I love about software engineering. Whenever a bug pops up, I get to don my black hat and switch to detective mode. Yeah, it’s like being a detective, because a good bug hides deep inside layers of code and you have to look for clues and rely on your instincts to catch it. The bug becomes an illusive killer wreaking havoc in the production systems. But the real delight is when you catch the culprit, and you follow its trails, watching how it started with a simple change, how it kept hiding but growing and finally oozed out completely metamorphosed.
I’ll tell you about an old bug which I had found really interesting. This part will be very technical, so reading further is upto the reader’s discretion. In one of our applications, people used to enter text in text box and save it. We got complains that users were observing random “???” popping up in saved textual data, even though they had never entered it. Upon checking, we did find a lot of such instances in our data. Something was adding these extra question marks, but who or rather what?
For tracking purposes, we kept history of changes in our data. This is a fairly common practice in software industry. We started to check when these random question marks started appearing. After a bit of trial and error, we narrowed it down to a Monday. We saw that after that day, saving data resulted in addition of these question marks. Upon further investigation we found out that these characters were not random, these were actually special characters like [‘, ’, á, ö], which were getting replaced by “???” and that the data was already corrupted when it reached our backend systems. We were able to simulate this in our QA environment as well. One of the engineers found a solution by adding a new encoding header in our API calls. We tried it and our special characters started getting saved properly, but still some features were not working like searching for text which contained special characters. We still didn’t know what caused this issue out of the blue as before that Monday everything was perfectly fine. We did have a production release during the weekend before the Monday, but we hadn’t released anything related to text search and storage, so we had ruled it out. The bewildering thing was that we didn’t observe this issue when we simulated the exact same scenario in our local machines! Even without the API header fix, everything was working normally in our local, search and save both. I decided that it has to do something with the release, all clues were pointing to that unfateful day of release. So, I started carefully checking each and everything that was modified in that release, and then it hit me!
We had pushed a change where we had added request and response interceptors for API calls. We had to analyse the request before it was handled and take some actions and then let the request go through. However, there is catch with interceptors, once you read the request data, you consume it, it gets removed from the request object, and the application would receive an empty request. So, in the code the request data was set back into the request. Now, when you intercept the request, it is read as a stream of data, and then it can be converted into a byte array, or String or some other object. In the code, the stream was converted to String, and then string was again converted to data stream and set back into the request and this was where the magic happened. When the stream is converted into String and back, the system doesn’t know what kind of encoding needs to be applied for the conversion, so either you can specify the encoding in the code or it uses the default encoding, and this encoding depends on the Operating system where the program is running! Windows uses UTF-8 by default and Linux uses ISO. And this, my friends, was the culprit. That was the reason we were not observing the issue on our local machines as our machines ran Windows. We had to specify the default encoding for the JVM to be UTF-8 and voila, the issue disappeared, everything started working as usual.
You see, the lifecycle of the bug? It starts with a small change, remains hidden, keeps festering until it shows as a completely different issue. This, this whole investigation keeps me on the edge, the joy of finding small clues and finally uncovering the truth. This makes me love what I do.
I’m a creator!
Ok, let’s shift gears. What’s the basic human instinct that drives growth and evolution? I believe it’s creativity. Humans love to challenge themselves and push the boundaries of what they can achieve further and further wide. Creativity is at the crux of this rebellious sentiment. Whether it is the innovation of airplanes, of new age drugs and vaccines or even inter-planet travel, we are always striving to create. Think about what was happiness for you as a child. Did you build objects from lego set? Or, did you find new ways to create the paper airplane that flew the farthest? Or, did you create new music? Or, did you solve that math problem that nobody in the class could? Or, did you make the best art in your art and craft class?
The best form of satisfaction or rather happiness is the one you get by doing something on your own, with your own hands. The satisfaction is not immediate, it takes time, it feeds over your efforts and doubts, it evolves you and finally rewards you. That’s why humankind is so obsessed with creating AI and robots, coz the pinnacle of creativity is to create another human or rather a better version of human itself. When we devote our full efforts and build something, it gives us a sense of control, it makes you feel a bit like God, with powers to alter the structure and behaviour of the thing being built. I got a similar taste of power when I wrote a C++ program to read text in natural scene images in my sophomore year in college.
It was for a course project for a course which was nowhere related to image processing. I am a practical guy and I love applying the wallet-devouring premier education on to real life situations. I remember having a shiver of excitement run through me when I first successfully executed the program and out of the input image it produced a line of text. The text was not entirely correct, but the very act of building an intelligent program which could literally “see” an image and tell me what was written in it gave me a thrill like no other. It’s a kind of high, a fever pushing you to create and develop more and more. And I still feel the same thrill when I write a piece of program does something on its own. And this thrill has kept me going, the thrill of being a creator!
