Some notes on the genesis of AI-based plugin development.
I’ve tried to code audio plugins. I failed miserably- heavy mathematics knowledge is often required to make anything good, and I didn’t devote enough time to study. Despite my best efforts, my attempts resulted in a subpar distortion plugin that created more noise than I would have hoped.
Yes, I admit, OpenAI’s Chat GPT is much better at making audio software than I am. As a human, I just have the advantage of being able to compile it.
Where We’re At Now- The Beginning.
Programmers have begun to experiment with ChatGPT alongside others who use it for homework assignments and generating ad copy. DSP developers, another name for the people that make audio plugins, are among them.
An early example is this emulation of the famous Ibanez Tube Screamer, a distortion pedal famous in rock music. The plugin doesn’t sound like the original hardware, and its code is very basic. In fact, without the human, it wouldn’t have been able to run at all. However, the bot’s ability to hit the ballpark and provide the majority of the emulation is still impressive.
Its knowledge isn’t limited to simple distortion plugins. Youtuber Doctor Mix, a layman, prompted the chatbot and received some code for a simple sine wave synthesizer.
A similar issue arose here as with the pedal – ChatGPT couldn’t generate compilable code without human help. Meaning, it won’t run. Individuals with less than intermediate coding knowledge would find it challenging to create a plugin.
Watching the video, I cringed as Doctor Mix pasted code meant for C++ into a Swift app builder. Despite the availability of ChatGPT to assist with compiling, he was still working in the wrong direction. Good content, though, Doc.
Despite its achievements, ChatGPT still requires human intervention and expertise to make sense of its generated code. In the case of Doctor Mix, he was unable to effectively use the code written by the bot alone. Even with MIDI-specific code, he would still lack expertise in implementation and proper structuring.
ChatGPT has potential in creating plugins, but human intervention is still necessary for C++ development. DSP developers still play a critical role in ensuring that plugins meet specific requirements and function properly. Until we have AI that can compile and debug for us, humans will remain an essential part of the process.
Wait, What is a Plugin?
Plugins are tools used in music software to expand the user’s capabilities – they have become ubiquitous in the music industry over the last two decades. The Fabfilter Pro-Q3 Equalizer, for example, offers much more than just the ability to reduce or boost certain frequency ranges. Xfer Serum, a popular wavetable synthesizer, dominates the electronic music scene with its (once) unique features. Emulations such as the Softube SSL XL 9000 K-Series sound just like expensive hardware.
Representing the best of the industry, these plugins are developed by engineers at the top of their field. However, none of this code is open-source, which means that it is not readily available for chatbots like ChatGPT to use.
The Challenge of Finding Quality Open-Source Code
Commenting on the Tube screamer emulation above, Redditor Dave9876 writes, “No surprise at all that it just gives example code off the internet.”
Strong opinions. I agree.
ChatGPT may not be ideal for plugin coding due to the scarcity of high-quality open-source code in the audio development industry. This scarcity can make it challenging for a chatbot to find suitable code to use, especially since much of the good code is often kept behind closed doors.
GPT-3 is primarily trained on tutorials. Sources like books, Wikipedia, and StackOverflow provide textbook examples rather than specific use cases. Outbound links from sites like Reddit often consist of news blurbs or beginner walkthroughs. Github’s user-generated content may lack quality due to the absence of a vetting process beyond usage statistics.
The Importance of Human Intervention in Plugin Development
While AI like ChatGPT can generate code and offer innovative approaches to language modeling, it’s important to remember the vital role that humans play in the plugin development process. From understanding the nuances of audio engineering to being able to vet and test code for quality, humans bring a level of expertise and intuition that can’t be replicated by a machine. As AI continues to advance, it’s important to recognize the limitations and potential pitfalls of relying too heavily on automated tools in plugin development.
Overall, while the future of plugin development with AI is exciting, it may take several billion more prompts for ChatGPT to dream up VSTs named “Electric Sheep”. However, with its advanced communication capabilities, it offers a unique approach to language modeling that could lead to new and innovative plugins in the future.