BigSoft Limited

Google

Posted by on 01 Jul 2023 in Work

Job title

Software Engineer

Industry

Finance, Insurance

Duration

3 years

Location

Amsterdam

Awards & accreditation

Proved that the customer needed to give us more money - and they did!

More...

Multinational teams from across Europe and India. Software Engineer at Google, moving mainframe financial applications to the cloud.

The job involved building and testing the company product ("Converter") which took PL/1 modules (for the mainframe) with embedded SQL and converted them into Java. The Java modules would then run inside a Java Virtual Machine supported by a set of in-house built Java support libraries ("TechSupport") which handled the annotations, mappings, database access, etc in the generated code.  We were converting the last 3 major versions of Allianz Business System ("ABS") and the subsidiaries' specific overrides. I worked in the TechSupport team.

Day to day this meant running the Converter on the ABS code, modified with subsidiary code and running it through the simulator. This ran the tests in a VirtualBox environment and compared the results to the same run using the original PL/1 compiled modules instead. The PL/1 code was 500K to 11M LOCs depending on subsiduary, 60 years old and had had thousands of developers working on it over the years using many different styles, syntaxes and dialects of PL/1. As the converter came across a new scenario it would produce: invalid code that did not compile, code which compiled but crashed at runtime, compiled but produced the wrong results or code that was not compatible with the current version of TechSupport. We had to find the problem code, read the corresponding PL/1 code and work out what the correct Java should be and feed that back to the developers of the converter or update TechSupport ourselves. If the problem could be fixed by applying a patch to the PL/1 source pre-converter or a patch to the Java source post-converter then an assessment to determine the priority of adding it to the converter took place. We all got a nice feeling when we upgraded the converter as it meant we could drop a set of patches as maintaining all the patches as code shifted around was a bit of a nightmare! The goal was a Converter requiring no patches which converted all the code and produced the same results as the PL/1 run.

The process of checking the code was very time consuming. When I arrived it took about 12 hours to do 1 complete test run. By the time the project ended I had re-engineered the process down to taking only 1.5 hours by distributing the work across several VirtualBox nodes, using chained Jenkins tasks and inventing a branching strategy which allowed us to deal with differentials of changes when determining which task to run next.

The nature of the project meant that there was a lot of manually inspecting PL/1 source of different versions to see why the converter was no longer able to convert that module. There was an equal number of inspections that would happen on the Java side too as the converted code moved around. I wrote a side project using ExpressJs and ReactJs in order to simplify this process. Users could run the application locally and point it at the directories containing the different versions. It was more than just a diff application as it had different ways of searching for files, allowed bookmarks and could compare different subsidiary subfolders in the 2 main directories.

There were dependencies between the Converter version, TechSupport version and customer code versions so we needed to be very sure about our versioning, release procedures and documentation. I used Japicmp to ensure our semantic version numbers were being adhered to. The release notes had to contain the state-of-play i.e. which bugs had been fixed since the last version. I utilised GitLab APIs to query GitLab bug/milestone/etc information to do this and wrote a lot of smaller scripts to parse result files to build the final release documentation.

During the project I became the go-to guy for questions about Git and so when we had to split a 10GB repository into smaller pieces, remove directories of binaries that should never have been checked in while retaining all the history then everyone volunteered me for that job ;)

The project had expanded a lot since I started. We now had a Git repository for TechSupport, a different Git repository for each subsidiary, a host of versions which needed to be managed, our internal jFrog repository (that I managed), external jars released by the customer on an adhoc basis, peripheral software for generating release documentation and dealing with VirtualBox snapshots. We needed a new way of managing and releasing our code. I was in charge of defining and documenting a GitFlow which was specific for our set of projects. I produced detailed documentation of different use cases at pinnacle points in the project workflow. These described with diagrams how to interact with the source controller, when and where to create labels, branches and how they should be named. Workflow use cases went from what to do after you assign a bug to yourself, knowing where to branch from and what to call your branch in a simple situation. In a more complicated situation, creating a hotfix for TechSupport for just one subsidiary and ensuring that after each further release those hotfix patches could still be reapplied to make a tailored customer version to accompany each 'normal' version, until just time that we could upgrade all the customers on to the new major version.

I had demonstrated that my knowledge base was extensive and covered a lot of different and distinct technologies. As a result people started to invite me on to projects that weren't really anything to do with the TechSupport side of things. These included fixing an AngularJs program that someone had written ages ago for the customer but had left the company, re-writing the VirtualBox API layer to control taking snapshots, creating a JNDI module to interact with binary code, adding basic authentication to another webapp, noticing 3rd party jars had been updated, pulling them into our jFrog and updating settings files so that things were automatically upgraded, obfuscating the deliverables, etc.

There was quite a high churn of people on the project and so I often found myself mentoring new people and bringing them up to speed. At this point the Converter was finished enough that no more work was going to be done on it and it moved from a development project to a maintenance project at which point everyone was let go except for me. I was kept on for a little bit longer to handover to the maintenance team as I was the only person left from the original project and the only person who knew and understood the decisions we made at the beginning and why we made them. I imparted this knowledge on to the maintainance team and answered their questions about the history of the project.

I like computer art and so for fun I produced a set of graphviz diagrams to represent the nature of the data being used on the project - because that is fun for me :) In this case I graphed the interaction between all the modules i.e. which modules called which other modules. When I presented my findings at Friday Fun it showed that the customer was trying to get us to convert 2 source code bases for the price of one! It went quiet for a bit then suddenly there was a load more money from the customer!

Friday Fun was something I invented as a bit of fun after the Friday meeting. I demonstrated my art and the support applications I'd written and other people did things that they were passionate about, not necessarily about computing, just anything they thought would be fun. I've heard from one of my ex-colleagues that he set up Friday Fun at the company he worked at after this project :)