Refactoring as a Software Design Technique

This is the fifth episode in a series on writing a simple JSON parser in Kotlin. You can see the playlist here - https://www.youtube.com/playlist?list=PL1ssMPpyqocicj5gxqpPDA1iZu9OX3YBT and the code on GitHub https://github.com/dmcg/json

I had an interesting chat on Twitter this week in response to a question about whether people who swear by TDD are just not using the type system properly.

That’s an interesting question. It’s certainly true that developers in SmallTalk, where TDD was born, used tests to verify things that a static type system would make a given. But I’m a Kotlin developer, and Kotlin has quite a nice type system, so why do I have to write so many tests?

The discussion helped me realise that, for me at least, there are two types of systems. If I can look a problem and be clever enough to write types that describe it, then those types can help make illegal states unrepresentable and cut down on the testing.

In the case of the JSON parser that I’m writing at the moment though, I’m not that clever. Perhaps if I’d had a descent education I would know the advantages of a recursive decent parser and the types would be obvious, but I didn’t, and this is an experiment in using TDD to solve interesting and real world problems without preconceptions.

When I can’t just look at the problem and design types, then I start by adding an example in the form of a test, getting it to pass, adding another example test, getting it to pass, and continually refactoring, often introducing types, to keep the code simple enough that I can cope.

We left last week with code complicated enough that I don’t feel that I can add any more features. So today we’ll simplify, coincidentally by adding a type, and that will give us some breathing space to both add features and further simplify other parts of the code.

In this episode

* 00:01:43 Review the mess from last week
* 00:02:38 Refactor to simplify
* 00:04:54 Add a test to reveal a bug
* 00:06:09 Introduce a new state to represent the top level ground
* 00:11:44 Pull on the thread of good error messages to improve the design
* 00:14:38 Make literal fail faster on invalid characters
* 00:16:52 Check in work so far
* 00:17:05 Introduce ArrayGround state
* 00:20:53 Introduce SeekingComma state
* 00:26:59 Check in now only commas are allowed
* 00:27:15 Look for places where things don't look right
* 00:29:09 Eagerly parse array elements
* 00:35:22 Review

If you like this, you’ll probably like my book Java to Kotlin, A Refactoring Guidebook
(http://java-to-kotlin.dev). It's about far more than just the syntax differences between the languages - it shows how to upgrade your thinking to a more functional style.