Relighting with AI

RELIGHTING is an AI-powered tool that transforms the lighting and environment of images. It can alter lighting conditions, change backgrounds, and adjust the overall atmosphere of an image using text prompts, reference images, or light maps. This technology is useful in various creative fields by enabling experimentation, saving time in the creative process, and allowing quick visualization of different concepts. Its potential applications are endless.

However, the technology still faces challenges. It currently changes more than just the light, morphing faces, limbs, and other elements. Because of these drawbacks, I decided to test this AI feature with a humorous twist and embrace the morphed limbs.

My project started by "casting" this (Midjourney generated) corporate guy. You can just tell he's looking for an adventure.

I then prompted him in situations and poses I would need him in my story (using --cref, --sref, etc.)

Then I worked on the environment.

Once I was satisfied with what I had, it was time to use AI to relight.

As you can see below, when you place your character in a new environment, the light and colors don’t match.

With AI Relighting, you have two choices. Either you relight video footage or you relight a still and then, if needed, generated a video.

For video you could use IC-Light. It's far from perfect but you can see where we're headed with these tools.

For still images you can use Krea's new Scene Transfer or Magnific AI's Relight.

Relighting my corporate guy looks like this:

I’ll let you witness the fun I had putting this guy and his colleagues in different situations.

For those interested in other elements of AI creativity, let me show you how I like to prompt Udio and Suno for generating music.

If you have a style of music or a specific track in mind, I discovered that ChatGPT-4o is an amazing music producer. By chatting about music production with it, you will be able to prompt the AI with much better success.

“Da Funk” by Daft Punk prominently features the Roland TB-303, a monophonic bass synthesizer originally designed to emulate bass guitars. This instrument is well-known for its aggressive, screeching filter and onboard sequencer, which became iconic in the acid house genre. In “Da Funk,” the TB-303 is used to create a distinctive acid bassline, which is further enhanced by running it through a distortion pedal. This gives the track its gritty and memorable sound. In addition to the TB-303, Daft Punk also used the Roland SH-101 throughout their discography, including on “Da Funk.” The SH-101 provided the duo with a versatile and classic analog synthesizer sound that contributed significantly to their signature style.

As with anything generative AI, I can't promise the same results as me but I've had loads of fun prompting in this way:

Use a Roland TB-303 for the acid bassline, Roland SH-101 for additional synths, and TR-909/TR-808 for drums, program a 16-step sequence on the TB-303, adjusting filter settings for variation, run through a distortion pedal, create a basic beat with TR-909 and layer TR-808 elements, add sampled drum breaks for texture, add melodic lines with SH-101, using effects like phasers and delays, apply compression, EQ, and sidechain compression, add reverb and delay for depth.

Fun fact: I chatted with ChatGPT-4o about a specific section of a relatively obscure song from 10 years ago and it described it perfectly. I have no idea how this is possible.

So despite all the current limitations, RELIGHTING represents a significant step forward in AI-assisted image manipulation and continues to push the boundaries of what’s possible in digital image transformation. I hope this was useful.