METASKETCHING
AUGMENTED IDEATION RESEARCH PROJECT



Abstract

Metasketching (framework) explores the role of sketching as a thinking process, as an ideation vehicle.


About author
My name is Mike Jelinek and I am concept designer who worked on projects such as Terminator Dark Fate (Tim Miller/Jim Cameron) or Dubai’s museum of the future exhibition research (Tellart). Notwithstanding my industrial design background, I am passionate about ideation and human perception in the design context, which is a subject matter of this (Ph.D) research.

More info and contact: https://www.linkedin.com/in/mjelinek/



Introduction

We all do that. We all doodle, we scribble, we sketch. On napkins, post-it leaves, magazines; directly on everything and everywhere. We all love to fill handcrafted pages of beautiful Moleskin sketchbooks with our illustrations, which we tend to call sketches as well. However, wait. Are these still sketches? Also, what is the sketch, actually?

Let's take a brief look into what art theoreticians say: "A sketch (ultimately from Greek σχέδιος – schedios, "done extempore") is a rapidly executed freehand drawing that is not usually intended as a finished work. A sketch may serve many purposes: it might record something that the artist sees, it might be used as a quick way of graphically demonstrating an image, idea or principle or it might record or develop an idea for later use." - So far what Wikipedia offers to us. Great definition for a start. Unfortunately, somehow misleading.

First of all, it claims that sketch is not meant to be seen as a finished artwork. Yet, it is going to be consumed as a piece of complete visual information.

Secondly, as a form of a real life drawing exercise, it undoubtedly represents itself as a finished work - like the reflection of the artist's opinion on the observed object or scene.

However, the concept of the sketch as a representation of an emerging idea makes much more sense. It removes the focus on the artefact itself and positions the sketch instead as a process of forming and communicating the idea in general. Such a process is often called "ideation" and sketching is its major component.



The role of sketching as an idea generator

Bill Buxton's book "
Sketching user experiences" (Focal Press, 2007) defines sketch using following attributes: Sketch is quick, timely, inexpensive, disposable, plentiful, while it contains clean vocabulary, it's made with distinct gestures and minimal details and appropriate degree of refinement. The goal of sketching is to suggest and explore, rather than confirm. The ambiguity of sketch offers diverse interpretations and contributes so to further idea development.

It's fascinating to observe how the first set of parameters provides a clear set of limitations when it comes to generation of ideas - when we look at their opposites. We might need to dedicate too much time to the drawing, and obviously, we need to find out the right slot during the day and we will also most likely over-think it too much. Moreover, while we aim for that "perfect one" piece, we give up experimentation, and then, using that expensive Moleskin sketchbook for sure requires the best what we can do.

We can probably agree that drawing on recycled paper or a napkin with a cheap ball pen somehow stimulates our creativity, as it removes the fear of destroying an expensive media irreversibly, and so it allows us to experiment more and generate more ideas. However, how ambiguity contributes to creativity? The answer is surprisingly simple: The sketching is not a one-way process. We only don't draw what we see mentally. As our brains are hardwired to find understandable forms in abstract shapes anywhere in nature and around us, our unfinished strokes work back the same way and create opportunities for our mind to re-interpret them.

Let's not stop there: The creativity may even multiply when we sketch in groups. There also exists a popular exercise based on sharing half-done sketches across the group and allowing others to continue with them. Just the physical presence of the audience makes a significant impact on creativity. It is the common fear of being watched. Having people around us when we focus on any activity positively influences our performance, it could be a positive one when it comes to sketching. It has happened frequently to me that during public sketching demos, my ideas were more diverse and original than those created in solitude. The reactions of the viewers are both consciously and subconsciously affecting our decisions, as we continually attempt to read how our actions are interpreted. It sounds uncomfortable and disturbing, but it seems to work.




Sketching in digital realities

The purchase of the computer and a drawing tablet may not represent the best case of being inexpensive and disposable. However, once we make that investment, the medium itself embodies an infinitely disposable for the cost of the electric energy we burn by running our PC's. The digital medium enables us to experiment, duplicate, repeat, draw over and over, without any fear of losing the medium itself. The medium is never consumed entirely, and that allows us to take risks impossible to take in physical reality.

Unfortunately, the computer interfaces do not provide the same level of gestural freedom (and fidelity) that we use to know from sketching on a piece of paper. Wacom stylus or Apple Pencil might be very close to pen and paper experience, but the computer's UI and operating systems add additional complexity to our interactions.

Yet, there is finally a virtual reality medium, which has unprecedented potential to remove the UI entirely and enable utterly new sketching experiences. For example, tools such as GravitySketch liberate users from the office desk space and allows them to express naturally while being spatially aware, and to move through their ideas in 3D space. Especially in so-called room size setup, gestures are made through the motion of the whole body, which significantly improves visual perception and improved ability to express graphically in an unrestricted manner. VR might be simply the ultimate sketching (ideation) tool, as it combines the benefits of digital media with intuitive interfaces, exceeding the real-life experiences.

Last, but not least, comes the diversity possible only in digital media. While the traditional, understand analog sketching happens only on given 2d media, creators can interact with various types of other media, such as photographies, images in general, 3d objects, generated geometries, and even motion and sound. All of that without losing the freedom of gestural expression.


IMG_3050

Sketching with GravitySketch during the VR Ideation workshop held at SVOTT, Czech Republic, 2019



The rise of ideation

Without any doubts, we all probably agree that ideation sometimes called the concept phase plays an essential role in design development across multiple fields of creative disciplines. From architecture, through product design, up to concept art, sketching represents the primary form of idea formation and communication. Although sketching may play various roles, as we've already learned, it mostly serves as a starting point for further development, and it sets the foundation for final execution and production. Which seems to be a crucial point: The production processes recently receive massive attention of the software developers, who aim to automatize them.
Thanks to AI and machine learning, artists and designers spend less and less time on rendering, 3d modeling, unwrapping, or data optimization. Moreover, it doesn't stop at technical levels. Tools such as style transfer, brush engines, and filters such as Akvis, or complex AI-driven painting generators help an artist to reduce their effort on generating multiple visual concepts required for production decisions.
So, what will remain when the AI will make everything? Well, I doubt the AI will ultimately replace artists from the creative process. I believe in intelligent tools helping us with the tedious and repetitive processes, allowing us to spend more time on research and ideation. In a way, it seems to be predictable that industry requirements will shift from strong technical skills to the ability to generate ideas - that means that ideation skill may become a leading factor when hiring creative staff.
The need for more creative professionals who can generate ideas on request will inevitably lead to questions such as: How can we teach and nurture the ideation skills? How can we support them, and how they could be catalyzed?


IMG_0391

Participants work at "Ideation workshop" held at XCMG, Xuzhou, China, 2018



Real-time engines and ideation

As mentioned above, it’s the diversity that makes digital media so attractive and efficient when it comes to ideation sketching. However, the majority of tools suitable for the idea generation serves in a very narrow manner. Such limitation comes from the design of those tools, which must obey their purpose. Adobe Photoshop may support import of 3d objects, but the scene management can’t compete with fully dedicated 3d content creation tools, such as Autodesk Maya. Also, tools such as Blender3d or zBrush allow creators to paint over the 3d surfaces, but for the cost of dealing with complex UI.
The rise of universal 3d real-time engines may change this situation dramatically — their primary purpose results in a design which doesn’t limit any use, nor media type. Also, the ability to run code as part of the user experience increases the possibilities of using real-time engines as an optimal and universal ideation tool.




Metasketching

While exploring possibilities of VR sketching, an idea to use it mainly as a particular solution for the ideation as mentioned above process emerged. Inspired by Aristotle's Metaphysics, I have started to call this concept Metasketching, where "Meta" represents the world "beyond."

Following examples are rapid experiments with GravitySketch VR - all under twenty minutes, which are sometimes taken to Luxion's Keyshot, where they are rendered with a toon shader, to maintain the dominance of gestural strokes, rather than pretending 3d perfection. So, they carry most of the attributes on Bill Buxton's definition of sketching, they are gestural and intuitive, while they offer an added value: The output goes beyond 2d image, beyond illustration - because Metasketching happens in 3d space, the output is naturally 3d file too. Also, having 3d data early in the process is worth of gold.

The Metasketching series kicked off late December 2018 and will continue as part of the "Augmented ideation" research project in upcoming years. Additional examples are available on
Metasketcher Instagram account, or Artstation under the same name and hashtag.


Screenshot 2019-05-15 at 08.02.25

Metasketching series, Michal Jelinek, 2018



Metasketching modular ideation toolset

Although real-time engines such as Unity3d or UnrealEngine are mainly helping users to create and build standalone applications, they can also be as they are, as a creative application itself. A user can import various types of 3d models, images and combine them with a code, and they can be a perfect universal environment for ideation across creative disciplines.
In an example, user can import a rough design model of a vehicle, add simple VR rig, and present early concepts as a 1:1 immersive experience. Then, a simple code can add the possibility to annotate the presented data a share it with the team.
Such a concept could be developed even further on: What if we don’t stop with the annotation module, and allow creators to add various prefab modules dedicated purely to ideation? They might be represented as simple as natural painting tools on one side and with content intelligent sketching tools supported by AI and machine learning on the other one. Any of these modules could be paired with imported geometries and VR rig, which together with camera/screenshot module would allow creators to interact with geometries and generate new ideas or concept iteration in an unprecedented, innovative way.



Additional resources and reference

Metasketching VR tutorial at Gumroad:
https://gum.co/eaLm

Vinod Goel - "Sketches of Thought," MIT Press, 1995
Bill Buxton - "Sketching User Experiences," MK Elsevier, 2007
Steven R. Holzman - "Digital Mantras," MIT Press, 1994
Lev Manovich - "The Language Of New Media," MIT Press, 1995
Jay David Bolter, Richard Grusin - "Remediation," MIT Press, 2000
Nick Montfort, Noah Wardrip-Fruin - "The New Media Reader," MIT Press, 2003
Jaron Lanier - "Dawn of the New Everything," by Henry Holt and Company, 2017 AlchemyDraw - http://al.chemy.org, Karl D.D. Willis, Jacob Hina, 2010




Metasketching research progress updates, documents and publications

2019:
- Initial dissertation proposal:
link
- more soon



Metasketching related events


13. 06. 2019 - STUBA, Noc Architektury Bratislava, "Ideas for sale", short talk about ideation in design process
14. 09. 2019 - Splash Conference in Prague, Czech Rep., "Introduction to Metasketching" (Lecture) -
link
19. 09. 2019 - Source of Creation Conference in Krakow, Poland, live Metasketching demo












Previous experiments:


Particle draw experiment with Unity3d:
Audio-reactive particle brushes




B_Body: Visual representation of echolocation - sensory alignment experiment.
Project's mini site:
http://www.horseville.net/b_body/




B_Body: Scenarios