Home Computer graphics Apple invents Sketchbook app for iPad that will allow user to merge sketches with real-world photos

Apple invents Sketchbook app for iPad that will allow user to merge sketches with real-world photos



Today, the United States Patent and Trademark Office issued an Apple patent application relating to computer graphics, and in particular to sketch-based placement systems and methods of computer-generated graphic objects in computer graphics. real world photos using an iPad.

In some cases, a user can populate their computer-generated room by selecting virtual objects from a pre-existing library. However, this limits the customization of the computer generated room.

Apple’s patent covers various implementations, systems, and methods for sketch-based placement of computer-generated graphic objects (sometimes also referred to as “virtual objects”, “graphic objects” or “extended reality (XR) objects”) in a computer -graphic-generated parameter (sometimes also called a “virtual environment”, “graphical environment” or “XR environment”).

One method is performed on an iPad using one or more cameras. In different layouts using the device.

The method also includes: obtaining a three-dimensional (3D) model using the input that corresponds to the sketch of the candidate object; generating a computer generated graphic object using the obtained 3D model; and causing the presentation of the computer generated graphic object with the imagery obtained using the camera (s) of the device.

In some implementations, the iPad is configured to present computer-generated graphics content and to allow optical transparency or video transmission of at least part of the physical environment (for example, including a table) on the screen.

As shown in Fig. 4A, the iPad also displays a # 410 content creation interface and a # 420 tool panel on the screen. According to some implementations, the content creation interface is configured to detect / receive user input such as sketches or lines with an Apple pencil held by the user or user touch / finger input. Depending on some implementations, the tools panel includes selectable tools configured to modify one or more characteristics of user inputs retrospectively and / or prospectively, such as line thickness, line color, line type. , fill color, fill texture and / or the like.

In some implementations, in response to detection of user input in the content creation interface that matches a sketch of a candidate object, the iPad is configured to obtain a 3D model based on the sketch of the candidate object and present a computer generated graphic object in the computer generated graphics setting # 425 based on the 3D model. As such, computer generated graphic objects are placed in the computer generated graphics setting based on user sketches directed to the content authoring interface.

As shown in Fig. 4B, instance # 450 of the first computer-generated graphics presentation scenario associated with time T2 shows a computer-generated graphics object # 475 (for example, a 3D palm tree) displayed in computer-generated graphics setting # 425 in response to detecting user input # 465 (for example, a sketch of a palm tree) in the # 410 content creation interface.

Apple’s patent FIG. 6 illustrates a flowchart representation of a placement process based on sketches of computer generated graphic objects.

(Click on the image to enlarge)


For details, see Apple’s patent application number 20210383613.

Since this is a patent application, the timing of marketing such a product is unknown at this time.

10.51FX - Patent Application Bar