The article focuses on the visual generalization in computer programming by example. In programming by example (PBE), also known as programming by demonstration, the system records actions performed by users in the interface and produces a generalized program that can be used later in analogous examples. A key issue is how to describe these actions and objects selected by the user, so the system can determine what kind of generalizations are possible. When a user selects a graphical object on the screen, for example, most PBE systems describe the object in terms of properties of the underlying application data. If the user selects a link on a Web page, the PBE system might represent that selection based on the link's HTML properties. The article explores a different and radical approach, using the visual properties of the interaction elements themselves, including size, shape, color and appearance, to describe user's intentions. Only recently has the speed of image processing made feasible PBE systems' real-time analysis of screen images.