Selections and Masks, Part 1: Selection Basics

In a nutshell, a mask is a channel (an Alpha channel) in the image, with the full 256-levels of 8-bit grayscale available. The various levels of gray represent what parts of the image are within a selection – and to what degree.

In a simple selection (without feathering or anti-aliasing), a pixel is either inside the selection or outside the selection. If a filter is applied, it will have an effect on pixels inside the selection, but not on those outside the selection. By using a mask, you can assign up to 256 different levels of how much that filter will be applied.
In the figure below, a circular select was made in the center of the image and the filter Texture> Mosaic Tiles was applied. Pixels within the selection were changed; pixels outside the selection were not changed.

In the next figure, a mask was created with a radial gradient and the same filter with the same settings was applied. Note how the filter fades out as it moves farther from the center of the image. In the corners of the image the filter is not applied at all.

When we look at the Alpha channel from which this selection was made, we can see that the white area allows the filter to be completely applied, and the darker areas restrict the filter. In the areas that are black in the mask, the filter is not applied at all, the image is completely protected.

With the image also visible, the mask is shown as a red overlay. As you can tell from comparing the image below to the image above, the red areas of the overlay represent the areas of the image that are protected.

Looking at the Channels palette, you'll see that the mask channel (Alpha 1) is both visible and active, and the composite channel (RGB) is visible but not active.

So, now we know what a mask is (and Alpha channel) and how it functions (protects and exposes parts of an image based on a grayscale channel). Let's look at the mechanics of creating and editing a mask.

The easiest way to make a mask, in most cases, is to start with a selection. Use any of the selection tools and commands. Once you've got a selection, you can use either of two techniques: Quick Mask mode or the command Select> Save Selection (which will be discussed below).

With a selection active in the image, you can press Q on the keyboard or click on the Quick Mask button near the bottom of the Toolbox.

Note in the Channels palette that entering Quick Mask mode creates a channel with an easily-identifiable name. (Keep in mind that Quick Mask channels in the Channels palette are like Work Paths in the Paths palette: They go away automatically if you don't save them.)

With the Quick Mask channel active and visible, we can edit the mask using any of the painting and selection tools. When using painting tools, such as the Paintbrush, Pencil, Paint Bucket, and Gradient tools, remember that black protects the image, white exposes the image, and shades of gray leave the image partially exposed. With the composite channel visible as well as the mask, the red of the overlay represents the black areas of the mask. Painting with black adds to the masked area (which will show as red).

Using the Eraser tool or painting with white will remove red areas from the mask overlay, exposing parts of the image.

You can even use filters on the mask, just as you can on any channel. (Applying a filter to a mask can lead to very creative results, especially if a filter is then applied after exiting Quick Mask mode.) In this case, the Magic Wand was used to select the non-red (unmasked) area and the command Select> Modify> Border was used. With a border of 36 pixels, a radial gradient (black to white) was dragged from the center of the image.

Once the mask is ready, pressing Q again (or using the button to the left of the Quick Mask button) will return you to Standard Mode. You'll see a selection border that may or may not match the edges of your mask. Keep in mind that the "marching ants" selection border shows the extent to which pixels are at least 50% selected. In the mask channel, that includes all pixels with a grayscale value of greater than 128.

Note that the selection border shown below does not match that shown above. The preceding selection border was used only to modify the channel itself. The following selection border will be used to modify the image (the composite channel). It represents the extent of the mask.

In the next image, the command Select>Inverse has been used to reverse the mask, and then the filter Noise> Add Noise was applied.

Note that the area around the bear's head, which had been modified in the mask with the selection border, has less noise than other areas. Making the mask visible once again, we can see why.

Once you've created and edited you mask in Quick Mask mode, you may want to save it. You can do that by exiting Quick Mask mode and using the command Select> Save Selection (which opens the dialog box seen below).

You can also save a mask within Quick Mask mode by dragging the mask to the New Channel button at the bottom of the Channels palette. That creates an Alpha channel called "Quick Mask Copy." Note in the figure below that the copy's name is not in italics. That indicates that it is a regular Alpha channel and not a quick mask.

As mentioned above, when you have made your initial selection, you can use the command Select> Save Selection. This allows you to by-pass Quick Mask mode and simply create an Alpha channel directly from the selection. That Alpha channel can be edited just as a quick mask is edited, with the full range of tools and filters.