If I go to burger joint somewhere outside the UK, I’ll usually look at the price and mentally convert it into GBP. I guess I could use a calculator, but it seems slightly pointless. Traders use Excel all the time, but it doesn’t negate the need to be numerate. Would a bank want to employ a trader who had awful mental arithmetic? We have machine to do calculations for us, but we still need to have a way of understanding their output and reasoning whether it is right or wrong?
Justin Huang, CEO of Nvidia, was quoted recently, as saying we should stop telling kids that they need to code (see video here). The rationale is that LLMs will enable us to code using human language. I suppose to some extent you could call this talking his book. After all, Nvidia stock price has exploded in tandem with all the interest in LLMs and the way they need a lot of compute (preferably from Nvidia’s GPUs). To a certain extent, Huang has a point, but I think it’s not the whole story. Human language is by its nature imprecise compared with programming languages, and it’s a point my good friend Paul Bilokon, Rebellion Research’s Quant of the Year 2023 has made, as have many others. I think Grady Booch, one of the best known software engineers, said it best on his Twitter account recently in this tweet below, where he referenced Huang’s video:
AI will kill coding in the same way that compilers killed coding.
What kind of human language prompts would have sufficient detail, clarity, and precision to permit the creation of useful, evolvable, and maintainable executable assets?
We call those “programming languages”.
I guess I can understand why Huang made the point. After all, If you’ve ever used ChatGPT it can be truly impressive for generating code. Ask it a question on something Python related (and a specific question at that) and it will happily generate code. There is a massive amount of open source Python code online, so there is a lot of code that a model can be trained upon. Indeed, we have used it at Turnleaf Analytics to help us to generate some code for different parts of our large data pipeline and forecasting engine. However, once the code has been generated, then the real work begins. You can give ChatGPT feedback on mistakes in its output and hopefully it will improve it. Ultimately, you still need your understand the code it generates, because what you meant to say English may not be what is understood by an LLM, given the imprecision of human language.
If you have no understanding of the code, then it will make it difficult to evaluate the output, and debug any errors. So yes, ChatGPT can make a developer more productively, by speeding up the creation of code, and reducing the amount of code he or she actually needs to write. However, that code still needs to be checked and maintained. It’s like using open source libraries. Yes, you didn’t write them, but you still need to be prepared to debug the code, in case anything goes wrong (and yes it does!). We can talk about stuff like prompt engineering and like, but trying to write in a way that the LLM wants (and which might be unnatural for you from a human language perspective), kind of amounts to the idea behind programming, writing in a more precise way that the computer can interpret. As Booch implies, that is a “programming language”.
It’s exactly like the parallel of being numerate when using Excel. Using tools like ChatGPT, GitHub Copilot etc. do not remove the need to code, even if they reduce the need to code quite so many lines, and can massively speed up certain tasks.