Coding Methodologies for New Media Artists

/by

Introduction

Last week, I hosted Raunak Singh, as a coder-in-residence, at Xenoform Labs. I normally provide artist-in-residencies for new media artists throughout the year, but  Covid torpedoed the program for 2020 and 2021.  I am thrilled to be resuming some sort of programming for 2022 and am now experimenting with different models.

Raunak is a trained software engineer and we collaborated on this blog post, I designed for New Media Artists, working with software code for their own projects.

My background: I am an educator who now teaches code to Design and CS students, and a new media artist, who is proficient with programming code, and for new works, I’m learning new technologies. I used to run my own software company in my early 20s and am self-taught, with an undergrad degree in Political Philosophy and an M.F.A.

The problem is that new media artists need to know how to use a range of technologies in their projects. They may be strong in a single domain (electronics, Java coding, VR, etc), but don’t usually have the engineering experience that makes learning and applying new technologies efficient. These artists often must pivot to new tech as it comes out, in order to be current with their critical discourse.

What methodologies can we learn from software engineering and apply to an art practice, without becoming a full-blown engineer or getting stuck in the technical weeds? After all, artists want to focus on researching the developing the concepts and creating the art. The tech is the means to the end, just a tool.

But first, an introduction to Raunak, in his own words.

Raunak is a software engineer who comes from a civil engineering background. He enjoys working on inter-disciplinary projects that combine different areas of engineering to do interesting things. This has given him a lot of practice in quickly learning and building with new tools.

Raunak has worked with tools ranging from virtual reality, full-stack applications, and machine learning. His most recent work with Xenoform Labs was on NFT Culture Proof, a collaboration with Scott Kildall and Nathaniel Stern and was a large-scale participatory text performance on the Blockchain. Through this project, Raunak exercised his ability to dive into new technologies through learning how to build smart contracts, as well as frontend and backend components that interact with them.

 

Time is a finite resource

Most of this article is about practical time-management techniques, which has implications in both the short-term (a specific project) and long-term (generally becoming a better builder).

Raunak introduced me to the concept of the Planning Fallacy, one of the many cognitive biases that impact time management. After reading about it, I quickly realized that I’ve often been its victim.

To quote Wikipedia, “The planning fallacy is a phenomenon in which predictions about how much time will be needed to complete a future task display an optimism bias and underestimate the time needed.”

People often (1) think that they can do the task faster than others who have done it before you (2) don’t account for unknown delays or hiccups (3) don’t possess the necessary information for realistic estimates.

Experience will counter this bias and as Raunak points out, most junior engineers overwhelmingly have problems with underestimating the completion times since it’s hard to see future roadblocks since they haven’t experienced them.

So don’t be discouraged when finishing a project takes even 2 or 3 times longer than you initially thought it would take! Even (and especially inexperienced) engineers have this problem. Set expectations — if you know this is the case ahead of time, then stretch out your timeline and then when it does take 3 times longer, for example, get your fancy interactive electronics pushing live data to a website functioning seamlessly, then there are fewer surprises, and, you’re not brimming with frustration halfway through the project.

One thing that we spoke about was to ascertain when to outsource and know when to learn a new tool. With NFT Culture Proof, we hired Raunak to be the lead developer and do all the blockchain coding. It was beyond my level of expertise — I’m an expert coder (at least for an artist), but learning this new tech and properly deploying it was beyond what I could do in the 3-4 month timeframe we had.

Nathaniel and I scoped out this problem early and focused on securing a solid developer. Our first person fell through and she recommended Raunak, who worked out spectacularly well.

The problem with outsourcing work is that you don’t learn new tools. And, the more distance that you have from the actual software coding, the harder it is to (1) make serendipitous discoveries — there are often unintended good outcomes that you discover when you write code (2) maintain your code for future projects or changes (3) implement your vision, since you have to both find and trust someone to do it for you.

 

Spend 5-10 hours solely on learning the new tool 

This guideline helps  to manage technical debt, which is a concept that Raunak spoke about. The concept here is that there is a cost of additional rework caused by choosing an easy (limited) solution now instead of using a better approach that would take longer. To summarize: when you make a decision for a better short-term outcome at the cost of the long-term.

Spending just 5-10 hours on learning a new tool can result in net gains down the project timeline. An example of this that I’ve seen is learning how to use GitHub, the standard source-code archiving and sharing tool. Just about every coder uses it to manage their projects. I teach it in my Interaction Design class. It’s notoriously confusing.

In about 2-3 hours, newbies can learn how to use this tool. And the more you use it, the easier it gets in terms of workflow. GitHub enables you to backup your projects, revert to older versions and have a cloud-based storage solution. I have about 100 source code repositories, some public, some private for artwork I’ve made, dating back to 2005. I even have an intro to GitHub video here

Too often, I’ve seen new media artists working with custom JavaScript, Arduino code, or something else and they have multiple versions of the project — old semi-working ones with names like “TalkingShell_final_final_3”. It’s a mess of confusion and creates its own sense of anxiety.

The trick here is, as Raunak points out, that when learning a new technology, you don’t know how long everything will take. This makes it hard to make decisions like “should I spend time writing this code that will make this other code easier to write”?

This is why a guideline of 5-10 hours is helpful. In Raunak’s case, with NFT Culture Proof, he learned the basics of Solidity, which is a smart contract coding language for the Polygon blockchain that we used. He spent time with an Udemy course on the topic and then developed the project timeline.

That 5-10 hours can also help you out with the decision to outsource or not. Perhaps it’s going to be something you can bite off with some limited technical knowledge, or maybe you’ll figure out that WTF, this is really hard and beyond what I can do, as I did with any sort of smart contract coding. That time will not be wasted.

 

What technologies should you use?

Ask a trusted colleague. What you find online through Googling won’t be nearly as valuable as what someone in your own field, another artist would recommend doing. For example, if you are doing Arduino development on a complex project, I can recommend which development environment to set up based on your specific needs.

The Arudino IDE, for example, is fine for setting up something quick and easy, but for a more complex embedded project, I’d recommend Visual Studio Code with PlatformIO. This provides color-text options, context-specific lookups of functions, debugging and so much more. But, it takes a few hours to set this up and sort through the project management details.

How long will the technology be around? Languages like Javascript or Python will certainly be around in 5 years. Will the Polygon blockchain be relevant? Maybe. Others are more transient. When I worked at the Exploratorium, for example, I used the Eclipse IDE with Proclipsing to run our Processing projects. It was powerful, but at some point along the way, the Proclipsing extension stopped being updated and this left us all with projects that were near-impossible to update. The popularity of a tool also means that you will likely find more resources (e.g. Stackoverflow posts) on how to use that software tool.

Check out at the popularity of such tools. Will it be easy to go back to the more stable system? For example, with Visual Studio Code + Platform IO, I could port this project back to the Arduino IDE very easily. Since VS Code is a Microsoft product, it’s likely to be around for a while anyhow.

How hard is it to learn? If after 5 hours of mucking around, if the new technology is just too damn difficult, then that is probably time to think about outsourcing the project to a qualified engineer. Is it just that far beyond your skill set? Are you completely disoriented after a few hours or do you have a sense that this software tool will be your friend?

How is it suited to your needs? Don’t try to muscle something around that isn’t well-suited for your project. I have a close friend, not a coder, for example, who uses the After Effects scripting language to generate his projects. That’s fine for video work, but when he started talking about a fun project called Biden Bingo, that generated bingo cards for different states during the 2020 presidential election, well that’s where I stepped in with Processing and solved the problem much more quickly. Make sure the tool in question is actually useful and not a circuitous route to solve the problem.

How much time should you spend researching vs building? 

As Raunak pointed out: with software engineering, it’s likely that someone has already implemented something you are trying to do. Successful engineering involves learning when to look online for libraries or other people’s code and when to implement your own. On one hand, it can be a huge timesaver if you find that one library or Stack Overflow Post that does exactly what you need to do and it fits perfectly with other components of your project. On the other, this holy grail is often hard to find, and code which looks like it is useful often requires doing acrobatics around it to make it fit into your other code. A general rule is to familiarize yourself with common libraries used by the new technology, and to always do a couple of quick searches on how other people have accomplished tasks you are working on when working with a new tool. In a lot of ways, searching for code and discovering what’s out there is experience itself in getting to know a tool.

 

You don’t know how much you can do with the time you are given if you don’t know how long things take

Rank project necessities from highest priority to lowest priority, e.g. do you need a completely accurate visual sensor or do you want to focus on an aesthetically-pleasing UI. Most of the time, some features don’t get completed, so it will be helpful to bump the lowest priority stuff down the chain. Once again, this seems obvious in some ways, but many don’t do it. Having this sketched out before you start a project, helps since what often happens is that you get emotionally attached to the work you’ve already sunk into a project and make poor decisions. Maybe the least important things have become more vital in your work-addled brain.

Timelines are also important in balancing long term growth with project completion. Most people have an idea of how much time they want to spend on a project, and how much time they have to learn other things. But it’s hard to make decisions like this when you don’t know how much time things take.

This takes us to the classic Donald Rumsfeld quote:

…because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know…it is the latter category that tends to be the difficult ones.

Right-wing war mongering aside, there is a chunk of wisdom for us to chew on here, which is simply that we don’t know what we don’t know.

First, try to get those unknown unknowns solved as quickly as possible. Don’t leave them for the second half of the project. Overestimate your timeline. Do the best you can.

Raunak’s advice, re-echoing here, is to take a short course or work through a book to learn what you don’t know, then you can re-cast your timeline. Courses or books are great for unknown unknowns.

Build what you don’t know first, then you can handle known unknowns. For me, for example, there is was a lot of JavaScript coding I did for NFT Culture Proof and I had to make various forms and other things that I had never done before, but I know JavaScript reasonably well and have worked with it before, so was confident I could solve these problems easily.

Some parting tips:

  1. Iterate on your design often. It’’s hard to keep a design fully in your head.
  2. Don’t get stuck in the “I can’t code” mental loop. It is a self-fulfilling prophecy. While some people have a more natural aptitude for code structures, anyone can learn to code in a “good enough” way for art projects.
  3. Identify bottlenecks, and design around them. There are often multiple solutions to the same problem. Determine the specific components that you are most concerned about working with, and come up with potential backup plans if you find that it’s too complicated to make it work. For example, in the NFT Culture Proof project, we initially wanted to generate our SVGs in Solidity, but we switched to doing that in JavaScript when that proved too difficult.
  4. Try to have multiple ways of accomplishing things. If something doesn’t work like it should or if it’s taking too much time/energy, then it’s always great to have some other technique to fall back on.
  5. Break down problems into bite-size pieces. It’s like doing work on your house. If you look at everything at once, it’s easy to get overwhelmed. If you work on just the shelving unit, that task can get done in a weekend and then you can move onto the next.
  6. Focus on handling one unknown at a time if possible. Don’t try to implement a smart contract and the UI for it in one go. It’s much easier to divide and conquer. Focus on a single component (e.g. the smart contract) first, and then, distill your work into as many simple tasks as you can (e.g. first deploy a trivially simple smart contract first, then work on how the smart contract token balances are stored, and then add transferring tokens between users).
  7. Know when you’re at the point of diminishing returns. If it’s 11pm and you can’t figure out a bug in the code, time to shut off the computer and work on it with a fresh can of Club-Mate in the morning.
  8. Prepare yourself for success. Figure out a workflow that is organized and soothing. For example, at the end of most days, I put together a to-do list for the next day. My brain is sharper in the morning and this is when I want to jump in and get shit done. Doing this is a good way to close out the work day and be focused on the next day.

 

Thanks Raunak for your insights!