Creating a Good Program

Apparently, if you want to be good in your field, you should not focus on solving the problem quickly but to find the right problem first. You find the right problem by asking the right questions. This seems to be the fundamental advantage of most experience: you have seen a similar problem before and you know the question to be asked. So, the important rule when writing a good program is not necessarily to stick with any sort of framework or techniques but to be humble enough and to ask the right questions.

What constitutes a good program? First and foremost it must solve an important problem correctly. Then, it must also be fast enough in execution and development. It must be maintainable and easy to be extended. How to achieve all of those points is not an easy task but it is a task that can be solved by asking the right questions. This is true for any programmer as well for anybody from the business side who ordered the production of a new application.

In the following, I list a broad collection of questions one should be asking. I will not provide any definite answers on them as they mostly depend on the concrete circumstance. But having the right questions will more quickly lead to a satisfying solution. The questions are loosely grouped first into principles, specific topics and then according to the timeline of writing the program. Let’s begin!

Principles

Wrong assumptions are the root of all evil

The worst errors usually stem from wrong assumptions. There will always be gaps in business requirements or in the user stories. The human mind tends to cover up any holes that it encounters in its mental map. A true professional spots those assumptions and tries to verify as many of those as possible before any major work is performed.

Think about and understand the business process

Business process means here in the broadest sense the process that actually adds any value, e.g. the pricing of a structured product in order to sell it to a client. A programmer has to know the problem (digitalisation of a business process) that needs to be solved. This comes back to the assumptions: if you know the process to be solved, you will ask better questions and rely less on implicit assumptions.

One can only understand this process by talking to the involved parties as much as possible. One might even be able to find out if there is already a solution e.g. like an excel sheet or manual process. This can greatly help in designing test cases.

Test your code

Usually, towards the end of a project, somebody from business will step by and ask, how to ensure that new releases are still correct. Nothing calms them quicker than leading them to a terminal window, running the tests and pointing to all the green lines. “This is how we make sure that everything is correct!”

But don’t just write test for the sake of having tests. Use it to test and verify your assumptions. Add new tests as you encounter new bugs since those indicate wrong assumptions.

Keep things simple

While it is always intriguing to test out the newest and hottest trends, avoid the urge and leave the play time to your hobby projects. Keep your professional work as simple as possible. Any code not written is code without a bug! And if you need to extend the functionality later on, it’s easier to augment something simple than to rewrite anything complicated.

Documentation

Keep the documentation as close as possible to the code. First, the code should be as clear as possible. Next, add comments where you need to add context and reasoning. Write it where it is not obvious for somebody else (or for you in six months) what is going on. Anything that is more general like overall goal or how to develop the code should go to a README file at the root of your code repository. Make sure that somebody else (or you in six months) can take the code, run it, build it and deploy it.

Deployment

One important aspect of your program is on how the users will interact with it and how you will provide the program. So you have to answer a couple of questions here.

• Is the program supposed to be interactive or is it run as a batch program?

• Is the interaction pattern synchronous or asynchronous?

• Does the program have a graphical interface or is it run in the terminal?

• Is the program used by power users or normal users?

• If it should have a graphical interface: native or a web app?

• Does it need to be redeployed regularly? If so, this calls for a web app.

• Do you need authentication and authorization? Do you need an audit trail?

• How much do you need to log? Do the logs need to be made accessible and easily searchable? Do you need to know about performance metrics?

• How fast does the program need to be? Do you expect an increase in volume?

• How many instances do you need? Do you need a redundant setup? What about state in this case?
  In general, try to avoid state as much as possible. Make interactions idempotent.

• What is the deployment policy? Are there release cycles? Or can you do continuous deployments? How will hot fixes be deployed?

• Do you have databases involved? Do you expect changes in the schema? How will you handle them?

Data and Domain Model

If you want to quickly assess the “IT situation” in a team or organisation simply ask the relevant person to draw a data flow chart. It’s shocking to see how little sometimes people know about what is going on in the code or in the broader setup of services. So focus on understanding all those loose ends, such that you can tie them nicely together.

Data sources and sinks

Identify all the possible sources for any incoming and outgoing data: users requests, events/messages, configuration files, databases, or any other static data.

Classify how much control you have over all the data sources and sinks. Do you control the schema? Do you control the generation of the data? How structured is the data? What happens if a data source becomes unavailable? Do you have any fallback? What happens if you cannot write to a certain sink? Is this an error?

Domains

Often, there is something like an inner domain model and an outer domain model. For example, when pricing an option, certain values like the strike might be expressed in terms of absolute money terms (1 CHF, 10 CHF etc.) which is fitting for the outer domain model that is talking to the user. But internally, it might make more sense to talk about strike levels (strike divided by the spot): 50%, 100%, 150%.

I used to be very strict about the differentiation between the internal and the external (API) model. However, these days I am more lenient about it. So, instead of having for example an inner and an outer ticker symbol, it might be enough to have just one. Just make sure that the API model (which is basically a contract with the user/client) is not changed without purpose and intent.

Also the outer domain should depend on the inner domain and not the other way around. Keep the inner domain as pure as possible. It should contain the pure business logic and becomes thus very testable.

This diagram shows the most common elements any program will have. Some might be duplicated, some might be missing entirely. The data flow starts with the requests and goes clock-wise.

Errors

Think about the possible error conditions. Most of the time they can be classified into either invalid user input or missing resources (data sources and sinks). How should the user be notified about those errors? If you have a nice split between an inner and an outer domain, keep any validation and checks in the outer layer such that the inner layer can safely assume to be working with correct data.

Technology stack

Only once you have answered most of the questions above you can seriously think about the technology stack to use. In general, the problem domain should express itself as easy as possible through the programming languages chosen. If you only need to quickly filter a lot of text data, it would be a total overkill to write a C or Java program if a small awk script would fit perfectly and be more manageable. Further on, weigh how crucial the program is for the organisation it is programmed for. If it is mission critical, keep it more conservative. A more conservative technology stack is one where you have a big community, a large ecosystem, extensive documentation and well established frameworks. If the program is not mission criticial, you can be more audacious and maybe use the program as a show case for introducing new technologies.

Programming Flow

Once you have a clear picture of the context and of the purpose of the program you can actually start writing it. Of course, you should still be on the lookout for any changes in assumptions you or anybody else in the organisation has made. As these changes always pop up as the solution (or rather the problem) becomes clearer, one should start programming where changes are the most costly ones.

Inner domain

Start with the inner domain and implement the logic of the business processes and models. Organise the code such that all data is being passed in. Keep the code as pure as possible (i.e. without side effects like writing to a DB or so). This helps to write short and useful unit tests for your code. Now is the time when you put all the gathered knowledge from above into code and where you really start to learn the ins and outs of the business domain. Is there anything unclear? If so, go back to business and clarify any open points. You will be left with a very robust core on which you can build your application.

Interface

The next step is to design and create a first implementation of the interface, no matter if it is a simple batch program with a CLI interface or if it is an intricate graphical user interface. If the interface is target towards other users, pitch it as often and early as you can. It will give you already valuable feedback and the users will feel involved and taken seriously. Fill the interface with simple mock data to make it more realistic. Again, validate any assumptions that you had so far and see if the users can see everything they will need later on. You might even need to fix some things in the inner domain. Do it now as it is still cheap to do so.

Connecting the inner and the outer layer

Now, connect your inner layer and your outer layer to obtain an interactive prototype of the program. Do not yet connect to any data sources that you don’t have full control over if possible. Mock that data (e.g. customer data, market data etc.). Again, show the program to the target user group. Check if the interaction feeling is fast enough or if you need to optimize anything.

Deploying the application

You should have by now a prototype that is interactive. It’s time to implement the whole deployment pipeline. Automatize as much as possible. Do not forget a mechanism to do automatic rollbacks. You want to be able to deploy a new release to be tested with little work but a lot of confidence thanks to all the automatic tests. Even if you just release to a development/testing environment, you should handle it as if it were a production release. Once the program is deployed make it available to your users so that they can start playing around with it and showing it off to other people. As they play around with it, you will gather again a lot of valuable feedback. Also, let them know if you have done small improvements. Communicate frequently.

Connect to your other data sources

Finally, connect your program also to all the necessary data sources and get rid of any mocked data provider so far (you can keep them of course in the code for your testing). Make sure that you cover the cases if a data source becomes unavailable. How will this be communicated to the user? Can you use any fallback data? Can you cache data for a while to increase speed and stability? What if a data sink becomes unavailable? How should the program work in such a case?

Finalization

By now you should have a working program that provides business value to the users. Gather all the feedback from the user acceptance testing and implement any upcoming change requests. While they become more costly now than before, if you have followed the instructions above, it will be much easier to change something as the program will have been build with the user and the business process in mind. A clean separation of the inner and the outer layer will make changes more manageable. Is something not good with the interface? Simply change the outer layer and the inner core will stay the same. Is there an error in the core? Simply change it and be confident that the user interface will not be greatly affected as the data flows per se are well structured.

Conclusion

The big principle when producing anything of great value is communication. This is especially true in programming. A programmer’s task is to help the user better perform his tasks and duties. To achieve this, the programmer must observe and understand the users and clients as much as possible. He must leave all assumptions at the door and be curious as much as possible to understand the business process and its context.

From the context one can learn which data and services are already available and how the solution will fit into place. By being aware on what can be controlled and what not, what parts of the code are “pure” and what parts have side effects, the program allows to be easily extensible and changeable through its life cycle.

While this article has only scratched the surface of many areas, it can still serve as a good check list and starting point for programmers or anybody from the business side interested in the process of developing a program. It raises all the possible questions that must be asked if one wants to successfully implement a program.

Is there any topic you want to know more in detail? Let me know in the comments!