REGISTRATION CLOSED !

From VFX to AI engineering: an Interview with Felipe Pesántez

Interview by Kseniya Serebrennikova

You majored in Computer Science — how and when did you start doing VFX? Why did you choose this career path?

Yes, I majored in Computer Science, as all my life since I can remember, I liked to code and deal with numbers. However, growing in South America, I saw that at that time, everything software-related was focused solely on web development. It was basically just some HTML and that’s it. One time one of my teachers started talking about 3D modeling, and at that same time a new program in 3D animation was introduced at the same university. I switched to it and spent a few years studying 3D, but I never finished it because freelance jobs took over.

You quickly transitioned to senior roles, overseeing VFX production and carrying a great deal of technical work — what helped you achieve that? How did your background in computer science influence this shift in your career?

I think what helped me the most and opened so many doors was the fact that I always wore the 2 hats so to say. On the one hand, I was proficient in the technical side of things. On the other hand, I had a hands-on experience in 3D/CG production, a.k.a. the creative part. Indeed, my background in Computer Science helped a lot, because I knew quite a few things about computer architecture, OS, servers, etc. Every company and studio valued that skill set. My work involved a great deal of mentoring: checking tech stacks, overseeing production pipeline, keeping track of performance, and more.

Now that your career mostly revolves around programming and engineering, what are the most important skills you learned as a VFX artist that helped you become a TD? And on the contrary, what are some of the skills that are exclusive to technical positions that require a bit of extra work from those with a background in VFX?

I was never too much on the artistic side of things, even when I was doing what you see in the final shots of many movies. I worked on it mostly from the technical point of view. It’s a lot like building a video game, so you need to understand the performance, profiling, etc. Here, seeing everything in detail is essential, keeping track of all the underlying technicalities is key. But at the same time I dealt a lot with the purely artistic things like concept art, lighting, perspective, anatomy, design in general, and this is another skill that many studios saw as something a bit unique. The main thing to keep in mind is that you are using a computer at the end of the day, so you need to know how it works. You need a solid understanding of computer algorithms, architectures, what happens in its memory, etc. You need to be mindful of how you use a computer, how you treat the CPU, whether you overload the RAM — all of that is key. VFX shots and games can scale massively and if you’re not doing it right the entire production suffers.

As someone who is an ML engineer and an instructor in ML courses, what are some things you wish more students knew before approaching ML? How could studying ML benefit current VFX artists?

As I mentioned before, knowing how to use a computer is essential, so you should know basic computer science, understand algorithms, and be familiar with a programming language. This means not just knowing a language, but actually understanding what the computer is doing. Obviously, AI and ML are must-have tools for any industry: at this point, it’s like using a computer to do your job. Imagine it like this: back in the day, when computers were just starting to be the hot new tool for animation, there was still this debate of whether you should keep animating by hand or use a computer. Now, pretty much nobody does hand-drawn animation anymore. And that’s how AI is nowadays: some still question whether or not they should use it, but it’s already everywhere.

Since you are a huge AI enthusiast, how do you use AI tools in your work as a tech lead? Could you talk more about your Animation Prediction tool for Blender: what was the process behind it, how do AI tools like this one impact the creative process in VFX/post-production, are you currently working on some more AI tools and features for VFX?

I develop AI as a Tech Lead, I don’t just use it. My job consists of creating models, training the models, model deployment, etc., and of course dealing with all sorts of data. Also, as every other company, we explore, use, and fine-tune a lot of pre-trained models.

The work we do extends far beyond the VFX industry. We collaborate with a lot of companies and industries and create all sorts of AI-based tools from medical simulators to military training apps, AI tools for layer firms, airlines, and more.

The tool for Blender is just a small example, we could have done the same in just text data or any other DCC — that’s the logic here and in AI in general. This is a sequence prediction, so you can use it for any other type of sequence. For instance, you can predict the next words in a sentence, the weather for the next day, etc. It just happened so that we checked it in Blender with some animation. So there would be the same base for any kind of tool for any kind of industry. For example, if you have to predict where the forest extends in a 3D environment for a VFX shot, it’s no different from predicting where bacteria will spread in a human body. The logic behind those things is the same — it’s just up to you where and how you apply it.

Knowing what you know today, was there anything you wish you had learned earlier or done differently?

Honestly, I would have stuck to just programming and AI from the beginning, but that is completely personal because I like it more. Working at major studios and VFX houses was still an amazing experience: it got me travelling all around the world, I got to know so many amazing people, etc. But I also have to say that a lot, and I mean a lot of tech companies have game development and 3D as part of their pipelines, and it’s absolutely essential. So that part of my career helped me a lot and it’s definitely a big plus.

What is the best place to start learning and transition to tech field? Are there any books or courses you would recommend?

You have all the resources you want. Even on YouTube, you can learn Computer Science basics and practice algorithms. But for me, the best way of learning how to program is by creating games. There are full series on game development on YouTube, and you can learn so much, and it’s really fun. What’s more, once you get into it, you have access to all sorts of career paths. Start exploring different game engines like Unreal or Unity. I prefer Unity, but game engines as a whole are the best way to learn.

To wrap up, what word of advice do you have for those who are debating whether they should do engineering, coding, ML, etc.?

You’ll have a really good future with any role in tech, it doesn’t really matter. AI is a go-to tool everywhere, but to do AI, you need to know how to program. Along the path of learning how to code, you’ll see if developing AI tools is your thing. If not, you might find yourself in some other branch of software engineering.

Interested in building a career in a tech field? Check out Filipe’s course on Machine Learning. Learn all the AI essentials, from data structures to deep learning and beyond. And don’t forget to subscribe for more VFX and tech content.

Subscribe for Weekly VFX Inspiration + Free VFX Tools

By filling out this form you agree to receive email communications from Rebelway.

More Inspiration

SIDEFX CERTIFIED

TRAINING

PARTNER

AUTODESK CERTIFIED