Masking with calculations in photoshop
This tutorial is a continuation of the Masking and removing backgrounds in photoshop tutorial and builds on what you learned there. If you didn’t complete and internalized the techniques from there, please go and do that now or you won’t be able to follow along.
In the Masking and removing backgrounds in photoshop tutorial we learned how do grab a mask from the channels palette, and how to enhance it with curves, dodge and burn.
Sometimes you can get a jump start on a perfect mask, using the calculations command and masking with calculations.
We will use the same photo of the flower, just to stay in familiar territory, so if you have not download that photo, click on the image below a couple times and download the large version and load it into photoshop.
redflower channel masking photo
With the flower layer selected, go the to channels palette and from the image menu select the calculations command.
photoshop calculations command
Yeah, scary dialog time I know 
All the calculations command allows you to do, is choose 2 channels and apply a blending mode, it’s realy not that scary now is it?.
What you do is select the two channels that are most different from eachother by using the source 1 and source 2 Channel dropdown lists. For this image, that would be the red channel for source 1 and the blue channel for source 2
Now we want to apply a blending mode that creates a target alpha channel (Mask) with the most contrast. This is largely a trial and error kinda deal. You know you are looking for something that makes the background black, and the object you want to mask white, or the other way around. What’s black and what’s white doesn’t matter, since you can always just hit control+i to invert the resulting mask.
To get a feel for what will happen, go through the list of blending modes in the blending modes dropdown list and watch what happens. Also, for each blending mode you select, try checking off the Invert checkboxes and see what happens. This is why I told you it’s largely a trial and error process. Nobody is going to remember what each blending mode will do with invert on/off for all the modes since the effect they will have is dependant on the colors and luminace in the image and no two will be alike.
Many times though, the blending mode that is only available here in the calculations command.. Add is going to work, and that’s the one we will use for this tutorial.
Select the red channel for source 1 and the blue channel for source 2. Set the blending mode to Add, opacity to 100 (Opacity is rarely changed ffrom 100%), Offset to -200 and leave the Scale set at 1. Lastly, put a checkmark in invert for the Blue channel.
Hey, would you look at that, that looks like the start of a super mask.
Make sure the Result dropdown listbox is set to New Channel which is the default unless you changed it, and hit OK.
calculations channelmask 1
Now’s the time for some curves to bump up the white and blacks, so hit control+m.
Set the black input levels to 120 and the white input levels to 225, that’s what I found the best compromise. Feel free to play around and see if you can make it better and then hit OK.
photoshop calculations curves
Now we burn the grey areas like we already learned, and dodge the white areas to further enchance the white and black areas. The process is the same, but notice this time you don’t have the problems with the ratty edges, and the grey background and white flower responds much better to the dodge and burn tools, making the whole process a lot faster.
I’m going to stop with the images showing progressions of the mask, you should be flying through the mask by now anyway
Take care of the rest of the background by painting it over with black and paint the inside of the flower white.
Invert the mask to check if you got everything. Nope, still some stuff there. A quick dodge and burn and that goes away with no effort.
Invert the mask back again, control click on it to load it as a selection and go back to the layers palette. Doubleclick the background layer to turn it into a normal layer, and with the selection still active click on add layer mask.
Add a black background behind the masked out flower and notice that this time we have no background bleed either. Using calculations for masking made our masking job much faster and painless.
Now you are ready to read: Masking in the LAB Color space (Advanced)
