Dr. Ivona Tau is a generative AI artist from Vilnius, Lithuania, who makes use of neural networks and code as mediums in experimental images and movement portray. Her work blends artwork and expertise to evoke feelings via AI-driven instruments, remodeling analog and digital movie experiences by way of generative neural networks (GAN).
With 15 years of expertise in images and AI analysis, Ivona has earned recognition, together with the Greatest Award within the Digital Ars 2020 contest and being named one of many High 10 Ladies in AI 2022. Her work has been exhibited internationally at venues like Artwork Basel Miami Seashore, Christie’s New York, and Vellum LA, and is a part of museum collections akin to ZKM Museum of Up to date Artwork in Germany. Dr. Tau holds a Ph.D. in Synthetic Intelligence and is represented by MTArt Company.
On this interview, Ivona discusses her inventive course of, specializing in the stability between expertise and human emotion. She displays on the moral considerations round knowledge bias and the way she integrates AI into initiatives like Dominoes in Fluxus and Mythic Latent Glitches. Ivona additionally shares her ideas on the rise of AI-generated artwork and the necessity for significant, impactful creations in an more and more algorithm-driven artwork world.
Go to Ivona Tau’s MakersPlace Profile
Desire to Watch or Pay attention?
Brady Walker: Welcome again to MakersPlace Spotlights. Right this moment, I’ve with me AI artist Ivona Tau. Ivona, perhaps you may inform us somewhat bit about your self and your journey for any of our listeners or readers who aren’t aware of you and your work?
Ivona Tau: Sure, after all! I’m very comfortable to be right here. I really began in two areas on the identical time. I started with images round 15 years in the past, and I began in AI nearly 10 years in the past—nicely, perhaps seven or eight years at this level. I used to be evolving in each fields concurrently. Fortunately, about 5 years in the past, I spotted I might dwell in each worlds.
So, I began utilizing my experience in AI—my information of how you can prepare AI fashions—on my images archives. I continued capturing new images, then educated AI fashions on these images, exploring how this turns into a brand new medium—one thing very totally different from my apply earlier than.
BW: What do you get out of AI work that you simply weren’t getting with images and vice versa? I’m additionally interested in which inventive impulses are higher fed by the handbook strategy of images that may not be happy via AI.
IT: Yeah, that’s an ideal query. My purpose with AI isn’t to create good photos or replicate one other medium. What excites me is what AI can try this different mediums can’t.
The most important factor is working with giant units of knowledge, exploring frequent themes, and discovering relationships between issues that appear very totally different. AI can sift via hundreds of photos, figuring out recurring options, which is one thing we will’t do simply ourselves.
If you prepare networks in your knowledge, you get a standard understanding of your work that’s usually shocking—even aesthetically. I discover patterns in what I {photograph} and from what angles. However extra importantly, AI creates surprising mixtures, revealing relationships between issues that appear separate. It blurs boundaries and exhibits these in-between areas that fascinate me.
Utilizing machine studying seems like discovery. You could have a route, parameters, and knowledge, nevertheless it’s about experimenting and seeing what resonates along with your human instinct.
BW: The way in which you describe it, it’s nearly like utilizing the machine as a hyper-powered, naive baby—no pre-existing biases, simply, “Oh, that is like this, that is like this.” I discover that fascinating as a result of creativity has at all times been described as the power to attract connections the place others haven’t seen them. So, having a type of hyper-intelligent child draw these connections for you looks as if an effective way to seek out belongings you wouldn’t have in any other case.
I do know you had an curiosity in math and science that was separate out of your curiosity in artwork. I’m curious—while you went into learning AI on the PhD stage, and even earlier than, was that pushed primarily by an curiosity in math and science, or by an curiosity in artwork?
IT: My curiosity was pushed primarily by a want to grasp the world, as broad as that sounds. Each my curiosity in artwork and in science had been extra about seeing one thing than creating one thing. I by no means drew or painted or photographed from my creativeness. As a substitute, I attempted to see issues in plain sight that had been in some way hidden.
I regarded for fascinating gentle, conditions, and glimmers of on a regular basis life, looking for the gems inside them. This method was much like how I approached science—I needed to uncover what was hidden. I needed to grasp how issues work, the mathematical guidelines that govern the superb world we dwell in.
As an illustration, how does a pc realize it’s taking a look at a cat? It’s superb that simply via calculations, it may well try this. That’s what fascinated me—the need to be taught extra about how the world works and, later, how computer systems see the world.
I used to be drawn to AI analysis via the sphere of laptop imaginative and prescient. Once I realized about convolutional neural networks and the breakthroughs in analysis from 2010 to 2015, I spotted this was one thing I used to be deeply taken with. All of it got here collectively—whether or not utilizing a pc, my eye, or a digital camera, the drive was the identical: to grasp and see the world higher.
BW: The way in which you describe it, how has your aesthetic sensibility shifted or developed since beginning to use AI?
IT: It’s shifted lots. Once I’m capturing images particularly for AI, I search for various things in comparison with simply taking snapshots whereas touring. I begin seeing like a machine, specializing in patterns and visible cues. AI doesn’t perceive humor or irony, so I maintain these apart for my conventional images.
Initially, I hated the machine artifacts early neural networks produced. My purpose was to do away with them and replicate the attractive grain of analog images, which is my favourite. However after working with AI for some time, I developed an appreciation for these artifacts. It’s like how digital images made us discover grain and colour in a different way. Now, I discover myself drawn to the bizarre pixels and patterns AI can create.
So, my notion of magnificence has undoubtedly shifted, and I believe that occurs naturally. We’re continually influenced by new issues—exhibitions, books, social media—so our tastes evolve it doesn’t matter what instruments we use.
BW: How do you establish how a lot management to exert in your work?
IT: I’ve experimented lots with that. In some initiatives, you design a system and let customers generate one thing from it. You realize the potential outputs, however not the precise ones, so that you lose management. It’s a bit scary, and I spotted I favor having extra management—proper right down to how a group is offered.
Experiments are enjoyable, and I’ve realized lots from them, nevertheless it’s more durable to design a system that absolutely aligns along with your aesthetic. Dropping management means shedding a few of these vital choices that form the ultimate outcome.
Now, I favor to manage the whole lot, particularly the information—as a result of it’s crucial a part of the AI system for me. I additionally need management over how my work is offered—whether or not it’s printed on paper, projected, or one thing else. As you achieve expertise, you be taught what works in your artwork and what doesn’t, and also you wish to keep away from repeating issues that didn’t work.
BW: I wish to leap into a bit of the interview the place we discuss a number of of your current initiatives. Right here, we have now one piece from a collection referred to as Machine Hearted. As I perceive it, this collection explores how AI interprets human emotion. What was the method like, and what did you be taught?
IT: This collection focuses on how AI excels at many issues however struggles with human feelings. Duties with out clear coaching knowledge or actual labels are onerous for AI, and feelings are very subjective. We really feel complicated feelings that don’t match neatly into classes.
Most AI analysis round feelings focuses on seven fundamental labels: unhappy, comfortable, impartial, scared, and so forth. However in actuality, we really feel mixes of feelings—nostalgic and comfortable, but additionally grieving, or lighthearted however careworn. These complicated emotions are onerous to explain, even with phrases.
For this undertaking, I educated a text-to-image mannequin with longer descriptions to discover how AI interprets complicated feelings. AI struggles to grasp the deeper emotion and sometimes jumps to conclusions, utilizing shortcuts. For instance, it would interpret “vivid disillusionment” as vibrant colours, ignoring the emotional depth.
I experimented with totally different mixtures and curation—mixing summary work with figurative images to create a center floor. The undertaking was fascinating, and I spotted lots of the topics I used to be drawn to had been ladies. This led me to discover how ladies expertise and conceal complicated feelings.
BW: Was that inside Machine Hearted, or is it a separate collection?
IT: That grew to become a subseries referred to as She Was Machine Hearted, with 10 items I confirmed in Paris with Kate Vass. It was a conclusion to the Machine Hearted collection, a small subseries by itself.
BW: Okay, I wish to transfer on to Dominoes in Fluxus. You educated 10 fashions for this collection after which settled on one. Did that exact mannequin seize one thing distinctive the others didn’t? What made you gravitate towards it? Additionally, as a aspect observe, do you typically prepare a number of fashions in your initiatives and slender in on one?
IT: Sure. To start out along with your second query, I normally work with a number of fashions utilizing totally different parameters and mixtures to seek out what works greatest. I usually take present fashions and fine-tune them for only a few epochs to see the early information switch. I prepare many fashions for a short while to see which one higher captures the thought I wish to specific.
With Dominoes in Fluxus, I used to be actually looking for an intersection—it is a recurring theme in my work—between one thing synthetic, artificial, and human-made versus one thing natural and pure. I really feel this is without doubt one of the greatest conflicts we have now as people. We dwell between nature and civilization, drawn to the seaside but additionally to cities and expertise. I understand this conflict very strongly, which is why I come again to this theme usually.
For machine studying fashions, it doesn’t matter whether or not the information is pure, artificial, natural, or synthetic—the mannequin learns a steady illustration. I needed a mannequin that might fluently transfer from one world to a different, representing each natural and artificial ideas. The ultimate mannequin I selected did this the perfect—it transitioned easily between city, neon-lit landscapes and extra pure varieties, which was precisely what I used to be on the lookout for within the animations.
BW: Are you able to discuss concerning the narrative that emerges when viewing the five hundred curated photos collectively?
IT: The thought was that they’re like dominoes—every video connects at totally different factors, permitting longer tales or narratives to kind. I needed to depart it open, permitting viewers or collectors to create their very own narratives, similar to we, as humanity, create our personal tales. Will we begin with one thing artificial and scary, then transition to one thing pure? Or will we destroy nature and change it with synthetic lights as an alternative of pure gentle?
These summary narratives had been meant to be performed with, much like how I performed with level latency when connecting them to create longer movies. I left it open-ended, and I actually loved seeing how folks related the items and the outcomes they got here up with.
BW: One factor I realized via researching you, which I discovered fascinating, is that Lithuania was the final European nation to be Christianized. So, in a approach, Lithuanian mythology is the latest to shift from faith to mythology. Are you able to give me somewhat primer on Lithuanian mythology?
IT: Paganism continues to be robust in Lithuania, and it fascinates me extra now. I seen lots of my Lithuanian buddies have names tied to nature—flowers, nightfall, daybreak. It’s simply a part of the tradition, and nobody thinks a lot about it. Quite a lot of our traditions, like throwing amber mud into the hearth on New 12 months’s Eve, are pagan too.
We even have many pagan monuments and legends. The tales are sometimes darkish, but additionally stunning. As an illustration, there’s a story of a queen who falls in love with a snake king who rises from the ocean. Lithuania has a whole lot of forests, and plenty of gods from our folklore are mentioned to dwell in these forests. Every tree has a god, and there are gods for the wind, sea, and so forth.
The forest seems like dwelling to me. I spend summers in wild locations, and that connection to nature impressed Mythic Latent Glitches, the place I used the names of some pagan gods.
BW: Are you able to inform me concerning the course of behind creating this collection? I learn your weblog put up, however for the sake of the interview, I’m curious. What about the place the result differed out of your expectations, particularly with using AI and glitch strategies?
IT: I normally don’t begin with a transparent consequence however with a query—like, “How can I present this idea?” or “What occurs if I mix these items?” It’s like a science experiment. I set guidelines and see the place the method takes me, making choices alongside the best way about what matches with my apply.
For this collection, I began by coaching an AI mannequin and producing photos. Initially, I deliberate to have the mannequin working on the backend, however on FX hash, there was a reminiscence restrict, so I needed to manually curate the outputs, which gave me extra management.
I additionally used p5.js for the generative coding, incorporating glitch and mixing photos. The unique AI-generated photos are on my web site, and you’ll see how they had been reworked via the code.
Utilizing glitch was new for me, and I appreciated the parallel between mythology’s mysticism and glitch as this hidden ghost in computer systems. The method was enjoyable and unpredictable. I discovered one picture out of 30 that I appreciated after which went again to the AI mannequin to curate outputs that matched that picture. There was a whole lot of back-and-forth between AI and code all through the method.
BW: Might you stroll me via your each day workflow and the way you method your initiatives?
IT: That’s a tough query as a result of there actually is not any each day workflow. It’s at all times totally different, which I like. I work with a medium that’s continually altering. I’ve some issues I take pleasure in, like working with GANs and different AI strategies, so I’ve developed some strategies there. However that doesn’t cease me from exploring new issues—like coaching a brand new mannequin or studying up on AI developments.
I would resolve to attempt different images strategies for printing, like lumina sorts or cyanotypes—random issues that don’t appear associated to my apply, however they arrive collectively later. Quite a lot of my course of entails simply pondering—whether or not mendacity on the grass or sitting within the forest. It’s vital to take time to not do something—not create, write, or draw—simply assume.
There are some constant components: exploring new instruments, getting impressed, and pondering. I learn lots, particularly about older artists, since I didn’t have a proper artwork training throughout my images research. Even when artists like Bosch don’t visibly affect my work, some choices I make are based mostly on issues I’ve learn.
In abstract, there are constant components, however they occur in random methods, usually in random locations. I’ve a studio in Warsaw, however I additionally work in Vilnius or whereas touring. That’s the great thing about being a digital artist—you may work anyplace, even on planes and in airports. It’s hectic, however I prefer it that approach.
BW: Is there something you’ve learn not too long ago that made a particular influence on you?
IT: Currently, I’ve been fascinated by the Japanese photographer Daido Moriyama. I stumbled upon his exhibition in London late final 12 months—a retrospective of his work. He’s nonetheless dwelling and dealing, recognized for his gritty black-and-white road images. His photos are uncooked and soiled, but additionally very true, in a approach.
I’m fascinated by his aesthetics, though I don’t work in monochrome. He has this sense of eager for sure locations, which I really feel too—particularly with all of the touring and never dwelling in my dwelling nation. It’s a eager for occasions or locations I’ve by no means been to, which I can relate to. There’s a Japanese phrase for that feeling, genkai. It interprets into my apply as nicely.
BW: I really like Moriyama, although I admittedly have by no means performed a deep dive into him or his work.
IT: It began with one ebook for me, and now it’s 5!
BW: As an AI researcher, I’m curious to know your view on the climate-related results of the compute energy wanted for AI giants like OpenAI. Is there a sustainable future for AI?
IT: Sure, completely. It is a essential space of analysis, and I don’t assume it’s talked about sufficient. There are methods to scale back the power wanted for coaching AI fashions. For instance, fine-tuning fashions—beginning with one which already is aware of one thing—takes a lot much less time than coaching from scratch.
There are additionally strategies to shorten coaching occasions. For diffusion fashions, analysis focuses on decreasing steps within the inference course of. As a substitute of 100 steps to generate a picture, you are able to do it in simply two.
However foundational fashions, like these educated by entities like OpenAI, are totally different. Individuals like us don’t have the sources to coach fashions for months on tons of of GPUs. The issue is that this work is monopolized, and their coaching code isn’t public, so researchers can’t counsel extra environment friendly strategies.
If these techniques had been extra open, the analysis neighborhood might assist cut back computational energy and enhance issues collectively. However proper now, we’re locked out as a result of it’s all so closed off.
It’s regarding as a result of, seven years in the past, everybody was overtly publishing papers and sharing code, and even small researchers had been making contributions. It benefited everybody. Now, we appear to be heading in a troubling route.
Even in case you’re simply utilizing AI for enjoyable, it’s vital to be aware. Don’t prepare fashions longer than wanted—attempt to do the least computationally costly duties. Though smaller fashions use much less energy, it’s nonetheless vital to remain conscious of those points.
BW: Is there a priority about AI as a company software versus the standard conversations in our circles about AI as an inventive software? Now that Microsoft, Apple, and Google are leaping in, do you may have any considerations?
IT: My greatest considerations are totally different from what we regularly see in headlines. I’m not afraid of sentience or autonomy in these techniques. What worries me extra is how the whole lot is concentrated within the arms of some, and it’s turning into monopolized. One other problem is bias—the illustration bias within the knowledge these fashions are educated on. We don’t discuss sufficient about the place the information comes from or the way it’s collected.
Some artists are elevating these considerations, and there are initiatives about being extra open with knowledge and letting folks choose out of coaching datasets. That’s nice as a result of everybody ought to have the precise to say no to their knowledge getting used. This could possibly be solved by giving extra folks entry to coaching instruments, so anybody might prepare their very own fashions. However till then, it’s not one thing most individuals take into consideration.
We additionally want to make use of these instruments responsibly. Know-how now makes it simpler to do dangerous issues. You may as soon as rent a doppelganger to pretend a video, however now it’s attainable with only a few clicks. We must be aware of those risks, and whereas I’m undecided how you can clear up them—perhaps via regulation—we will’t return. We simply have to seek out methods to deal with it.
BW: It’s going to be an fascinating time shifting ahead. On a lighter observe, my final query is concerning the flood of AI artwork we’ve seen not too long ago, particularly with instruments like Midjourney and DALL-E. You scroll via Twitter or Instagram, and whereas it’s spectacular technically, it may well really feel tiresome when you may spot Midjourney artwork instantly. Who’re the artists utilizing these instruments in ways in which couldn’t be performed earlier than?
IT: My easiest distinction between AI artwork that excites me and AI artwork that doesn’t is whether or not it could possibly be performed with different instruments. If it appears to be like like a render or good {photograph}, it’s not that fascinating to me.
Artists like Kevin Abosch, Anna Ridler, Sofia Crespo, and Holly Herndon use AI in essential methods, addressing social questions, exploring the instruments, and discovering new expressions that weren’t attainable earlier than. Some are combining generative textual content and pictures in distinctive methods.
For me, it’s not about how stunning the result is however the story it tells. Sadly, AI artwork usually emphasizes stunning, quick artwork—what I name the “McDonald’s of artwork.” It’s like eye sweet: folks prefer it of their feeds, and it goes viral. The algorithms amplify this sort of artwork. I’d argue that extra fascinating AI initiatives don’t carry out as nicely on social media because the eye-candy ones. It’s a part of a much bigger problem of how AI is flattening our tradition, particularly on platforms pushed by algorithms.
