Lightweight Structured Editing

Lightweight Structured Editing with Direct Manipulation

Brian Hempel University of Chicago Grace Lu University of Chicago  and  Ravi Chugh University of Chicago

We present a structure-aware code editor, called Deuce, that is equipped with direct manipulation capabilities for automating structured program transformations. Compared to most typical refactoring environments, Deuce employs a direct manipulation interface that is tightly integrated within a text-based editing workflow. In particular, Deuce draws (i) clickable widgets atop the source code that correspond to subexpressions and other relevant features of the program structure, and (ii) a lightweight, interactive display of potential transformations based on the current selections. With these features, the user can quickly and easily identify, invoke, and configure various structured program transformations during the text-editing process. We implement Deuce and evaluate the resulting workflow within the context of Sketch-n-Sketch, an interactive functional programming environment for generating Scalable Vector Graphics. Through a series of examples, we demonstrate how the automated transformations in Deuce help reduce the amount of tedious and error-prone text-edits that arise in several programming tasks.

copyright: none

1. Introduction

Plain text continues to dominate as the universal format for programs in most languages. Although the simplicity and generality of text are extremely useful, the benefits come at some costs. For novice programmers, the unrestricted nature of text leaves room for syntax errors that make learning how to program more difficult (Altadmri et al., 2016). For expert programmers, many editing tasks—perhaps even the vast majority (Ko et al., 2005)—fall within specific patterns that could be performed more easily and safely by automated tools. Broadly speaking, two lines of work have, respectively, sought to address these limitations.

Structured Editing

Structured editors—such as the Cornell Program Synthesizer (Teitelbaum and Reps, 1981), Scratch (Resnick et al., 2009; Maloney et al., 2010), and TouchDevelop (Tillmann et al., 2012)—reduce the amount of unstructured text used to represent programs, relying on blocks and other visual elements to demarcate structural components of a program (e.g. a conditional with two branches, and a function with an argument and a body). Operations that create and manipulate structural components avoid classes of errors that may otherwise arise in plain text, and text-editing is limited to within well-formed structures. Structured editing has not yet, however, become popular among expert programmers, in part, due to their cumbersome interfaces compared to plain text editors (Monig et al., 2015), as well as their restrictions that even transitory, evolving programs always be well-formed.

Text Editing + Menu-Based Refactoring

An alternative approach witnessed by integrated development environments (IDEs) is to augment unrestricted plain text with automated support for a variety of refactorings (Griswold, 1991; Fowler, 1999; Roberts et al., 1997). This approach provides experts both the full flexibility of text as well as mechanisms to perform common tasks more efficiently and with fewer errors than with manual, low-level text-edits. Although widely used in practice, there are a couple of limitations. First, most of these automated transformations help with latter phases of development—refactorings (Fowler, 1999) are transformations that preserve program behavior while changing internal structure—but not earlier ones, as structured editing does. Second, as the number of refactorings supported by an IDE grows, complicated hierarchies of menus make it increasingly hard to identify, invoke, and configure a desired refactoring (Murphy-Hill et al., 2009; Vakilian et al., 2012; Mealy et al., 2007).

Our Approach: Text Editing + Direct Manipulation-Based Structured Transformations.

The goal of our work is to enable a workflow that enjoys the benefits of both approaches. Specifically, programs ought to be represented in plain text for familiar and flexible editing by expert programmers, and the editing environment ought to provide automated support for a variety of structured code transformations without deviating from the text-editing workflow. To help with all phases of the development process, some of these structured transformations should explicitly change the behavior of incomplete programs and others should help refactor complete programs. The net effect should be a programming workflow that is largely text-driven, but augmented with automated support for structured transformations (e.g. to introduce local variables, rearrange definitions, and introduce function abstractions) that are tedious and error-prone (e.g. because of typos, name collisions, and mismatched delimiters), allowing the user to spend keystrokes on more creative and difficult tasks that are harder to automate.

In this paper, we present a structure-aware editor, called Deuce, that achieves these goals by augmenting a traditional text editor with (i) clickable widgets directly atop the program text that correspond to subexpressions and other relevant features of the program structure, and (ii) a lightweight, interactive display of potential transformations based on, and automatically positioned adjacent to, the current selections. With these direct manipulation features, the user can quickly and easily discover and invoke a variety of structured program transformations within a familiar, unrestricted text-editing workflow. The name Deuce reflects this streamlined combination of text- and mouse-based editing.

Implementation in Sketch-n-Sketch.
Figure 1. Screenshot of Sketch-n-Sketch v0.6.0

We implement Deuce within Sketch-n-Sketch (Chugh et al., 2016), an interactive functional programming system for generating Scalable Vector Graphics (SVG). In Sketch-n-Sketch, the user writes a program (shown on the left half of Figure 1) in a core functional language, called Little, to compute SVG output (shown on the right). Unlike traditional programming languages, however, in Sketch-n-Sketch the user may interact with the output of the program—adding new shapes, relating and grouping shapes, and modifying properties such as as positions, sizes, and colors—and Sketch-n-Sketch synthesizes updates to the Little program in response to the user’s actions with the output. Whereas that work provides capabilities for directly manipulating the output of a program, our goal in this work is to provide capabilities for directly manipulating the code itself. We also demonstrate how domain-specific structured transformations can be co-designed with the Deuce interface, providing automated support that is tailored to a specific programming setting. Nevertheless, most of the features provided in Deuce work for arbitrary Little programs (not just those used to compute SVGs) and, thus, may be adapted and extended to code editors for other programming languages.

Contributions and Outline.

This paper makes the following contributions:

  • We present the design of Deuce, a code editor equipped with a lightweight, direct manipulation interface to identify, invoke, and configure structured program transformations while retaining the freedom and familiarity of traditional text-based editing. Our design can be instantiated with different sets of program transformations for different settings. (§ 3.1)

  • We instantiate Deuce with a set of structured transformations for several general-purpose prototyping and refactoring tasks. Most of these transformations are common to existing refactoring tools, but two transformations—Move Definitions and Make Equal—are, to the best of our knowledge, novel. (§ 3.2)

  • We implement Deuce within Sketch-n-Sketch, extend the set of structured transformations for a few domain-specific tasks, and evaluate the new structured editing features through a series of examples. (§ 4)

Our implementation and videos of our examples are available at Next, in § 2, we describe one of the examples to introduce the main ideas in Deuce. We conclude in § 5 and § 6 with discussions of related and future work.

2. Overview

We use an example in this section to motivate and summarize the workflow enabled by Deuce. The task, to write a program that generates an SVG design, is borrowed from Hempel and Chugh (2016). Throughout the discussion, notice how Deuce automates tasks that would otherwise be tedious and error-prone, and the user performs manual text-edits for tasks that require additional human insight and choice.

2.1. Motivating Example

Consider the task of writing a program that generates the Sketch-n-Sketch logo (shown in the adjacent screenshot), where two white lines atop a black rectangle reveal a lambda symbol between three triangles. We would like the program to be structured so that it can be reused to easily generate configurations of the logo with different design parameters, namely, the size, background color, and color and width of the lines. We will describe a sequence of text- and mouse-based code edits in Deuce that allows the user to prototype, repair, and refactor the code until it achieves the desired goal. The reader is encouraged to follow along with the example in our online demo, or to watch the video.

Phase I: Prototyping.

The user writes definitions for the background rectangle and the first line, between the top-left and bottom-right corners of the logo. To start, these shapes are both positioned at the origin (0,0) and stretch to (200,200). They are combined in (svg [rectangle line1]) to generate the first version to render; this code can be seen in the screenshot in the left half of Figure 2. The user confirms that the pieces look roughly as intended, and now returns to the program to replace the hard-coded (0,0) constants with variables so that the positions of both shapes can be modified together.

Rather than text-editing, however, the user can use the structured editing capabilities of Deuce to perform this task in a quick, and safe, way. The user holds down the Shift key, which causes Deuce to display clickable widgets when hovering over different parts of the program text. The user clicks on the two 0 constants that correspond to the x-positions of the top-left corners of the shapes, and Deuce displays a menu of potential transformations underneath the selected widgets. When the user hovers over the Make Equal tool in the menu, Deuce displays a list of candidate new variable names to add to the program. When hovering over each choice, Deuce previews the result of the transformation; in each case, the selected constants are replaced by uses of the new variable, which binds 0 and is defined at the innermost scope relative to the constants. The user chooses the new variable name to be x1, and the code is transformed as shown in the right half of Figure 2.

Figure 2. When the user holds the Shift key and selects the two constants (highlighted in orange), a context-sensitive menu identifies the program transformations that may be applied. When the user invokes the “Make Equal, New variable: x1” tool, the program is transformed to the version on the right.

Next, the user employs the Make Equal tool three more times, selecting the remaining pairs of constants that define the x- and y-positions of the top-left and bottom-right corners. Although happy with the four names Deuce has suggested (x1, y1, w, and h), the user wants to combine the definitions into a single line, because each of the names and values is short.

The user selects three variables to move—y1, w, and h—and the target position after the x1 variable, to indicate that the selected variables should appear inline after the x1 (shown on the right). The user hovers over the Move Definitions tool to preview the transformation where all four variables are defined in a tuple (i.e. 4-element list), and then selects this transformation.

Now, the user uses normal copy-paste to duplicate the line1 definition, renames it line2, and adds line2 to the list of shapes. This second line will eventually connect the bottom-left corner of the logo to its center, which will form the lambda symbol. To start testing, however, the user edits the start and end points to be (x1,h) and (w,y1), respectively, which generates the symbol “X” when run.

The user invokes Make Equal twice to relate the stroke color and width of the two lines with two new variables called stroke and w1, respectively. The user is not happy with the name chosen for the latter, so she clicks on the variable definition and uses the Rename tool to rename w1 to strokeWidth (the interaction is shown on the right), which automatically replaces all uses with the new name.

Then the user invokes the Move Definition tool to combine these two variables into a tuple (not shown). The user invokes Move Definition once again to move rectangle below stroke and strokeWidth (not shown). Having taken care to organize the code in a readable fashion, the user would like to define a variable to clearly identify that the constant ’black’ defines the color of the rectangle. The user selects this constant and the target position before stroke, and invokes the Introduce Variable tool (shown on the right) to add a new variable called fill in place of the string literal.

Phase II: Repairing.

At this stage, the user has become comfortable with the basics of the design, but is aware of two issues that must be addressed. The first issue is that even though the position of the top-left corner has been factored into variables x1 and y1, the relationships for the other endpoints depend on the values of these variables both being 0. To verify this, the user text-edits them to be 50 and 50, re-runs the program, and confirms that the lines do not “move” with the rectangle. Knowing what the intended relationships ought to be, the user text-edits the second endpoint of line1 to be (+ x1 w) and (+ y1 h).

Now, to snap the other line of the “X” to the top-right corner, the user must use these same subexpressions in the definition of line2. The user selects (+ y1 h) of line1 and h in line2, and invokes the Make Equal tool (shown on the right). Because these two subexpressions differ, the Deuce menu asks the user to select which of these expressions should be used in both places; the user chooses to use the sum expression. Similarly, the user invokes Make Equal (not shown) to replace the w in line2 with the (+ x1 w).

The remaining issue is that the second line needs to be half the length, so that it reveals the lambda symbol instead of the letter “X.” To do this, the user text-edits the coordinates of the endpoint to be (+ x1 (/ w 2)) and (+ y1 (/ h 2)), respectively. Viewing the output now reveals the lambda. Either with text-edits or the existing output-directed manipulation features of Sketch-n-Sketch, the user varies the values of the four positional variables, and visually confirms that the output continues to exhibit the intended lambda symbol.

Phase III: Refactoring.

At this point, the user has finished encoding all the desired relationships in the program. Now is the time to refactor the program so that it can be reused to generate multiple variations. First, the user selects the list of shapes at the end of the program and invokes the Introduce Variable tool (shown below left) to give it a name (shapes) outside the svg expression. Next, the user selects the definitions that contribute to shapes, and invokes the Move Definitions tool to place them inside the shapes definition (shown below right).

The top-level definitions are turned into local let-bindings, taking care of indentation and parenthesis delimiters that would otherwise require tedious, manual text-edits. The user uses Rename Variable (not shown) to change shapes to logo.

The final step is to turn logo into a function that is parameterized over the design constants inside the definition. Selecting the definition, Deuce shows an Abstract tool to turn several of the constants into function arguments. In Figure 3, notice how the use of logo has been rewritten to a call, with the appropriate arguments selected from their values within the definition. Again, this would be a tedious and error-prone manual transformation, as the connection between formal parameters and actual arguments are not syntactically adjacent in the program. However, this transformation is easy to automate with structure information. To create other versions, the user copies and pastes the function call and changes the arguments to each (shown in Figure 3).

Figure 3. Program to generate Sketch-n-Sketch logo, developed with a combination of text-edits and structured transformations.

To recap, the process of prototyping, repairing, and refactoring the program is a text-driven process, as usual, but with support for automating low-level, structural edits that are tedious and error-prone (e.g. because of typos, accidental shadowing, and mismatched delimiters). Furthermore, the tools can suggest useful information, such as the different variable names offered by Make Equal and Introduce Variable. As a result, the user spends keystrokes on the more interesting tasks that are harder to automate—arithmetic relationships in the design, the choice of names, and final decisions about whitespace and formatting.

3. Deuce: A Little Structured Editing Environment

In this section, we explain the design of Deuce in more detail. First, we define a core language of programs where the user can select structural features. Then, we describe a user interface that displays a set of active transformations that can be invoked. Finally, we describe the set of program transformations for prototyping, repairing, and refactoring tasks that are provided in our current implementation of Deuce.


The programming language that our editor supports is based on Little (Chugh et al., 2016), a core, untyped lambda-calculus with standard semantics. We make minor changes to Little, to identify structural features of programs to expose in addition to (i.e. on top of) the concrete syntax of the program text; this version of the language, called LittleDeuce, is defined below.

Figure 4. Syntax of LittleDeuce. Expressions, patterns, and let-bindings (also referred to as definitions) are all surrounded by orange boxes, to denote that they can be selected in the user interface. The target positions, denoted by orange circles, before and after expressions and patterns can also be selected in the user interface.

Notice that all (sub)expressions are labeled with unique ids ; the equation of let-expression is labeled as definition ; and all (sub)patterns are labeled as their origin (e.g. equation , lambda , or branch expression ) and path (a sequence of integer indices from the top-level pattern to the subpattern). Internally, to implement the subsequent program transformations, we also keep track of how expressions (resp. patterns) relate to the ancestor expressions (resp. patterns) that contain them. Each of these selection items is surrounded by an orange box, to denote that the item will be exposed to the user for selection and deselection. In addition, there are target positions, denoted by orange circles, before and after every expression and pattern in the program. Each target position will be exposed to the user for selection and deselection. This structure information is a hook for the user interface, to decide which program transformations to offer the user. Besides these cosmetic changes, the syntax and semantics of LittleDeuce are standard.

3.1. User Interface

The goals of our user interface are to, first, expose selection widgets—corresponding to the items and target positions in a LittleDeuce program—to the user and, second, to expose a set of structured transformations that are active based on the set of selections. So that the additional features provided by Deuce do not intrude on the text-editing workflow, we display selection widgets when hovering over the code box only when the user is holding down the Shift key. Hitting the Escape key at any time deselects all widgets and clears any menus, returning the editor to text-editing mode. This allows the user to quickly toggle between editing modes during sustained periods in either mode. When not using the Shift modifier key, the editor is a standard, monospace code editor with familiar, unrestricted access to general-purpose text-editing features.

3.1.1. Selection Widgets


Our current implementation draws an invisible “bounding polygon” around the source text of each expression, which tightly wraps the expression even when stretched across multiple lines. These polygons serve as mouse hover regions for selection, with the polygons of larger expressions drawn behind the (smaller) polygons for the subexpressions such that all polygons for child expressions partially occlude those of their parents. Because complex expressions in LittleDeuce are fully parenthesized, it is always unambiguous exactly where to start and end each polygon, and there are always character positions that can be used to select an arbitrary subexpression in the tree. Similarly, we create bounding polygons for all patterns and definitions.

When hovering over an expression (i.e. over its invisible polygon), Deuce adds a visible yellow border to the bounding polygon to indicate that it has become the focus. The interior of the polygon remains transparent, allowing the text behind to shine through, but the interior can be clicked to toggle selection. We use green and red borders, respectively, to indicate whether a click will select or deselect the item. We use orange borders to denote items that are selected but not currently hovered by the mouse cursor.

Target Positions.

In our current implementation, we draw circular icons for target positions on the single characters before and after the item begins and ends, respectively. This character is not always a space. For example, the target before one (shown on the right) appears over a space, but the target before three appears over the list delimiter. The target icon occupies only part of a character box, so any selection polygon behind it is still accessible. For example, in the screenshot below, the cursor is near the center of the selected target but not exactly over it, so the bounding polygon behind it (for the list expression) is the actively hovered widget.

As with items, we use yellow, red, green, and orange to draw target icons when hovering over and selecting target positions. We currently do not draw selected widgets for target positions after expressions; these positions can often be described using the target position before adjacent expressions.

3.1.2. Context-Sensitive Menus

Several program transformations may be active based on the items and targets that are selected. We design and implement a lightweight user interface for identifying, invoking, and configuring active transformations.


So as not to distract from the text-editing workflow, we auto-position a menu near the selected items, specifically, below and to the right of them. Currently, we choose not to take selected target positions into account when computing this position. In the screenshots in § 2, we often manually dragged the menu from the automatically-chosen position to make the screenshots fit better in the paper. The toolbox can also be pinned to a fixed location if desired, in which case its location persists across multiple Deuce interactions.

Safety of Transformations.

There are several different cases for the effect of a program transformation: (a) the transformation intentionally changes program behavior (e.g. Make Equal); (b) the transformation preserves program behavior (e.g. Rename); (c) the transformation cannot be verified to preserve behavior (e.g. Add Argument when not all call-sites are statically known); and (d) the transformation unintentionally changes program behavior, possibly causing it to crash (e.g. Move Definition above a dependency). We use white, white, yellow, and red colors, respectively, to mark these different effects. Note that sometimes the user may want to choose yellow or red transformations for various reasons during the development process.

Visual Previews.

To provide a quick explanation of each transformation, we display the transformed code (as well as its output) when hovering over an option. When a tool has only one option and that option is safe, the tool button itself is used to preview and trigger the transformation. In other cases, hovering over the tool button results in a second-level menu appearing next to the tool with the list of options, each with a description and a color to denote its safety (cf. the screenshots of Make Equal and Abstract in § 2).

3.2. Program Transformations

Active Transformations Selected Widgets
Tool Name (+ Options) Expressions Patterns (or Defs) Targets
Preserve Behavior for Reuse
Abstract (+ all constants or only named constants) 0 1 (or 1) 0
Merge 2+ 0 0
Preserve Behavior for Readability or Subsequent Editing
Move Definitions (+ renaming; dep. lifting; dep. inversion) 0 1+ (or 1+) 1
Introduce Variable 1+ 0 1
Add Arguments 1+ 0 1 in arg list
Remove Arguments at function definition 0 1+ arg 0
Remove Arguments at function call 1+ arg 0 0
Reorder Arguments at function definition 0 1+ 1 in arg list
Reorder Arguments at function call 1+ 0 1 in arg list
Reorder List Items 1+ 0 1 in parent list
Eliminate Common Subexpression 1+ 0 0
Rename at variable definition (+ new name) 0 1 0
Rename at variable use (+ new name) 1 variable 0 0
Swap Variable Names and Usages 0 2 0
Inline Definition 0 1+ 0
Duplicate Definition 0 1+ 1
Clean Up 0 0 0
Make Single Line 1 0 0
Make Multi-Line 1 0 0
Align Expressions 2+ 0 0
Change Behavior
Make Equal (+ choose suggested name) 2+ constants 0 0
Make Equal (+ choose expression to copy) 2+ 0 0
Reorder Expressions 1+ 0 1
Swap Variable Usages 0 2 0
Figure 5. General-purpose transformations in Deuce.

To demonstrate the extensibility of our user interface, we have implemented a variety of program transformations that are active depending on the current selection state; these are listed in Figure 5. While we believe these transformations form a useful set of basic tools for common functional programming tasks, we make no attempt to argue that these constitute a necessary or sufficient set. One benefit of our approach is that different sets of transformations—such as the refactorings for class-based languages proposed by Fowler (1999) as well as ones for functional languages proposed by Thompson and Li (2013)—can be incorporated and displayed to the user within our approach of selection widgets and context-sensitive menus.

We limit most of our discussion below to transformations that, to the best of our knowledge, are not implemented in existing refactoring or structured editing tools. Notice that, in addition to transformations that preserve program behavior (i.e. refactorings), we also implement several that intentionally change program behavior during the development process.

Move Definitions.

A common, mundane text-editing task is to rearrange definitions. While conceptually simple, there are several aspects that are subject to error, such as making sure not to break dependencies and making sure not to break scoping. Furthermore, particularly in functional languages where local-bindings can be arbitrarily nested and where tuples can be used to simultaneously define multiple bindings, there are many stylistic reasons for arranging definitions in a certain way. These choices are often in flux during the prototyping and repairing process, where definitions may be reordered to more clearly explain program dependencies and to ensure that the concrete layout fits nicely within the screen width.

The Move Definitions transformation takes a set of selected patterns and a single target position, and attempts to move the pattern and its definition to the target position. If the target position is an expression, a new let-binding is added to surround the target. Whitespace is added or removed to match the indentation of the target scope. If the target position already defines a list pattern, then the selected definitions are inserted into the list. If the target position defines a single variable, then a list pattern is created.

There are three cases in which Deuce provides the user options for how to correct an otherwise invalid transformation. In all cases, the user may choose the original invalid option since breaking the code temporarily may be the intention during the course of prototyping and repairing.

(a) Simply moving the definition of x on line 4 to line 2 would change the binding structure of the program. In addition to this option, so the second and third options use renaming to preserve the binding structure of the original program.
(b) Simply moving the definition of a3 above b would result in a use of a2 before it is defined, so the second option additionally lifts the definition of a2.
(c) For arithmetic constraints among expressions—such as the top-left corner, width, height, and bot-right corner of a rectangle— Move Definitions can move definitions above dependencies by rewriting the expressions.
(d) When selecting the patterns c and d and moving them before b, they are put into a single definition. When selecting the definitions c and d, the separate definitions are preserved in the new location.
Figure 6. Examples demonstrating four configuration options for Move Definition transformation.
Option: Renaming to Preserve Binding Structure

One issue is that the binding structure may change. For example, in (a), the uses of x in the expression (+ x 1) on lines 2 and 5 resolve to different definitions of x, on lines 1 and 4, respectively. Moving the definition of x from line 4 before y1 will result in a program that evaluates safely but with different binding structure. In this case, the Move Definition transformation provides several options: the unsafe option that performs the transformation without renaming which allows a binding to be captured, and two safe options that rename either of the definitions to avoid capture.

Option: Lifting Dependencies

A second potential issue is that the definitions would be moved before its dependencies. In the example in (b), a2 is defined to be a1, and a3 is defined to be a2. Trying to move a3 above b is not safe, because a2 is not in scope. The transformation provides two options: the unsafe transformation, and a version where the dependency, a2, is automatically moved as well to make the original transformation safe.

Option: Inverting Dependencies

A third situation is when the user may want to rewrite definitions so that a dependency violation can be avoided. In the top-half of (c), the program defines the top-left point (abbreviated to tl for space) to be (100,100) and the width and height to be 100 and 200, respectively; the location of the bottom-right point (abbreviated to br) is derived in terms of these parameters. Dragging the third definition above the second results in brx and bry being rewritten to constants that the previous expressions evaluated to (i.e. 200 and 300), and then width and height are rewritten to arithmetic expressions that preserve the original relationship (i.e. (- brx tlx) and (- bry tly)). In situations where there are constraints among expressions—particularly common in programming domains that generate visual output, such as a web application or data visualization—this transformation allows the programmer to vary the choices about which parameters are defined first in the program with constants and which are derived in terms of them. The bottom half of (c) shows how the new definitions can also be inverted, returning the program to the original.

Option: Flattening or Preserving Definition Structure

As listed in Figure 5, Move Definition can be initiated by selecting either one or more patterns (e.g. just the variables c and d in the top half of (d)) or one or more definitions (e.g. the definitions (def c "c") and (def d "d") in the bottom half of (d)). In the former case, the selected patterns are moved into the same list pattern in the target position; in the latter case, separate patterns are kept separate. This is useful for preserving stylistic choices of the definitions (e.g. the line length of each definition) as well as semantic properties (e.g. dependencies between the selected definitions).

Introduce Variable.

The Introduce Variable transformation takes one or more selected expressions and a target position, defines new variables for the expressions, and replaces the expressions with uses of the new variables. The transformation attempts to suggest meaningful names, based on how the expressions appear in the program. For example, if the expression 0 in (let x 0 ...) were extracted to a new variable, it would also have suggested name x. The transformation also takes into account how the value is used. For example, if the string ’black’ is passed as the first argument to a function (def rect (\fill x y w h)), were that expression ’black’ to be extracted to a new variable its suggested name would be fill (cf. the example in § 2). The transformation uses several other syntactic approaches like this in order to suggest potential names to the user. Although this approach cannot match the user’s intent perfectly, we believe this simple approach already does much better than a naïve approach (e.g. gensym1, gensym2, etc.) and reduces the number of situations in which the user must immediately think of better names.

Make Equal.

The Make Equal tool allows multiple expressions to be selected and proposes two kinds of transformations depending on the selections. If all expressions are constants, the transformation introduces a new binding at the innermost scope enclosing the constants, initializes it to one of the constants, and replaces all constants with the new variable. This has the effect of changing the program to make each of these values equal. The transformation attempts to choose a good name, as for Introduce Variable, but also proposes names by combining suggested names for the different expressions—for example, in § 2, the name x1_x was one of the suggestions because the constants being manipulated by Make Equal were passed to two functions, line and rect, whose corresponding arguments were named x1 and x. The user is asked to choose a name. The value itself is not as important—the intention is that the values vary at once by a single change—so the transformation does not ask the user to choose the initial value to use for the variable.

Alternatively, if not all selected expressions are constants, the transformation copies one of the expressions in place of all others (again, making them equal). Unlike the case where all selected expressions are constant, the key choice here is which expression to use, so the user is asked to choose which one to replace the others with.

Merge Expressions.

While prototyping, it is often convenient to copy-paste code and then make changes to the different clones as needed. Afterwards, it may be desirable to pull out the common code between the clones into a single function. The Merge Expressions tool takes multiple selected expressions and attempts to abstract them over their syntactic differences: any differing subtrees become arguments to a new function inserted into the program. The selected expressions are rewritten as calls to this function. To avoid suggesting unhelpful small abstractions, the Merge Expressions tool is displayed only if the resulting function is larger than a threshold (more precisely, if the number of AST nodes in the function body is at least double the number of arguments to the function).

Abstract Expression.

The Abstract tool transforms a selected expression into a lambda abstracted over constants within the expression body. The transformation provides two choices: (1) abstracting all constants, or (2) abstracting constants that are immediately let-bound to variables. The latter is a (simplified version of a) heuristic proposed by Hempel and Chugh (2016). We could add configuration options to ask the user about all potential constants to abstract; to keep this process lightweight, however, we propose only the two parameterizations and then allow the user to modify the result with tools for arguments (below). After the expression is rewritten, the variable it is bound to is tracked so that its uses can be rewritten to calls to the new lambda, with the constants that had been pulled out of the definition.

Add, Remove, or Reorder Arguments.

The tools to Add, Remove, or Reorder Arguments allow the interface of a function to be changed. Arguments may be added to a function by selecting expressions within a lambda and a target position in the argument list. The Remove and Reorder Argument tools allow the modification to be specified either at the argument list of the function definition or at a call-site of the function. All three transformations require call-sites to be updated in sync. Currently, we use a simple static analysis to track when a lambda is let-bound to a variable. If the lambda ever escapes this simple syntactic discipline, then we cannot guarantee that all function calls are rewritten appropriately. In which case, the transformation is marked as potentially unsafe (i.e. yellow). As in other cases with unsafe transformations, the user must rely on other mechanisms to ensure correctness (e.g. types, tests, viewing the output, or code review).

Reorder Items.

This transformation allows one or more (potentially non-consecutive) items to be removed and inserted elsewhere in the same list. The whitespace between each pair of consecutive elements is preserved in the transformed list, a detail that can often be tedious to ensure with manual text-edits.


A transformation commonly found in IDEs is to Rename a variable and all its uses. In Deuce, the variable to be renamed may be selected either at its definition or at one of its usage sites. In either case, as the user types the new name it is checked to ensure the name will not introduce collisions; if it would, the transformation is marked as unsafe (i.e. yellow) to indicate that a different name may be desired.

Swap Items.

Swap Variable Names and Usages can be used to correct the names chosen for two variables. The alternative is to perform the sequence of Renaming the first variable to a temporary, Renaming the second to the first, and Renaming the temporary to the second. A related transformation, Swap Variables Usages, is handy for when the definitions were correct, but their usages are the opposite of what is intended (e.g. mixing up width and height values).

Eliminate Common Subexpression.

This transformation allows several identical subexpressions to be replaced with a new variable, defined at the innermost enclosing scope. The same transformation can also be achieved with smaller parts: Introduce Variable for one of the expressions, and then Make Equal to copy the new variable to the other expressions.

Duplicate Definition.

Text-based copy-paste works especially well when entire, adjacent lines are copied. For expressions with smaller delimitations (within a single line) or larger, “jagged” ones (different positions across multiple lines), text-based selection may be more cumbersome. The Duplicate Definition transformation is a mouse-based alternative for copy-pasting an expression to a different target position.

Inline Definition.

This transformation replaces all uses of a selected variable with its definition. For convenience, multiple different variables can be selected and inlined simultaneously. As any definition being inlined may itself have variables, if any such variables are accidentally captured at their new locations then capture avoiding renamings are offered.

Clean Up; Make Single Line; Make Multi-Line; Align Expressions.

All transformations in Deuce attempt to handle whitespace reasonably. Even so, occasionally a transformation or series of transformations will result in a program with, for example, a long line of code. We supplement the whole-program Clean Up tool of (Hempel and Chugh, 2016) with whitespace reformatting rules to break long definitions into multiple lines and ensure that any multi-line definition is comfortably padded by a blank line before and after. This is currently the only transformation that requires no selected items—it applies to the entire program. We also implement several transformations to help format selected expressions, by adding or removing line breaks and indentation.

4. Evaluation

We have implemented Deuce within Sketch-n-Sketch, incorporated a few domain-specific transformations, and used the new system to develop several example programs.


Sketch-n-Sketch is written mostly in Elm (, a language in which programs are compiled to JavaScript and run in the browser. The project uses the Ace text editor ( for manipulating Little programs. (The second reason for the name Deuce is that it extends Ace.) We extended Sketch-n-Sketch to implement Deuce; the new version (v0.6.0, depicted in Figure 1) is available on the web. Our changes constitute approximately 6,000 lines of Elm and JavaScript code.

Numeric constants in Little can be annotated with an optional range—written 15{1-30}—to instruct the Sketch-n-Sketch editor to display a slider in the output to make it easy to change the number without text-editing. This feature is an example of “scrubbing” constants, a live programming feature described by Bret Victor (!/DrawingDynamicVisualizationsTalk) and Sean McDirmid ( We made one minor addition to the Little language: the option to mark a range annotation as hidden—written 15{1-30,"hidden"}—to keep the range information in the program while suppressing the slider widget in the output. Besides this, we inherit the syntax and semantics of Little unchanged.

4.1. Domain-Specific Transformations

To demonstrate how our approach can be instantiated with custom structured transformations, we have implemented several transformations (in addition to the general-purpose ones described in § 3.2) that are specific to the domain of SVG graphics in Sketch-n-Sketch; these are summarized in Figure 7.

Thaw/Freeze Number; Add/Remove Range; Show/Hide Slider; Flip Boolean.

In Deuce, the Add/Remove Range tool operates on constant literals to attach or remove these range annotations. New minimum and maximum range values are determined based on the current value. The Show/Hide Sliders annotates a range to be 15{1-30,"hidden"}, which keeps the range in the text but suppresses the slider from the output. This tool makes it quicker to toggle the sliders on and off (and, furthermore, preserves the possibly-edited min- and max-values to remain in the text). Sketch-n-Sketch allows numbers to be frozen with the annotation 15!, which tells Sketch-n-Sketch not to change this value in response to changes to the output. (Compared to the discussion of the Abstract tool before, the heuristic for arguments is to choose named and unfrozen constants.) As with sliders, it can be tedious to use text-editing to toggle this annotation on and off, so the Thaw/Freeze Constants tool provides an alternative. The Flip Boolean tool is a quick way to “scrub” boolean literals.

Active Transformations Selected Widgets
Tool Name (+ Options) Expressions Patterns (or Defs) Targets
Preserve Behavior for Readability or Subsequent Editing
Thaw/Freeze Number 1+ 0 0
Add/Remove Range 1+ 0 0
Show/Hide Slider 1+ 0 0
Rewrite as Offset 1+ 1 0
Convert Color String (+ RGB or Hue) 1+ strings 0 0
Change Behavior
Flip Boolean 1+ booleans 0 0
Figure 7. Domain-specific transformations in Deuce.
Rewrite as Offset.

Hempel and Chugh (2016) point out that there are often (at least) three different common ways to programmatically describe a positional attribute of a shape: with constants, with constant offsets from an anchor point, or with constant relative percentages with respect to a bounding box. When prototyping, it is often easiest to start by using constants and then later switching to one of the relative versions. The Rewrite as Offset tool converts one or more selected constants into offsets from a selected value. For example, in the expression (let x 10 15), rewriting 15 as an offset from x transforms the program to (let x 10 (+ x 5)). As with several other structured transformations we have discussed, this transformation is conceptually simple but can become tedious and error-prone to perform with manual text-editing when multiple numbers scattered across the program need to be offset and when the base values are not easy-to-remember numbers.

Convert Color String.

In Sketch-n-Sketch, as in HTML and SVG, colors can be specified in a variety of ways: RGBA codes, HSL codes, HEX, and color strings. The Sketch-n-Sketch editor provides special support for color attributes defined as “color numbers” which are are (essentially) just the hue component of an HSL triple; in particular, the editor displays a slider next to the shape to control this color value. The Rewrite Color tool converts color names (often useful for prototyping) into corresponding numbers, to enable the direct manipulation support for color numbers that Sketch-n-Sketch provides. This transformation is an example of how a custom program transformation can be used to complement other features provided by the IDE.

4.2. Examples

To evaluate our system, we used the new direct code manipulation features in Deuce to help implement five example SVG designs, including the one described in § 2. Two of these tasks are borrowed from Hempel and Chugh (2016); our examples demonstrate how to build the desired programs with a combination of text- and mouse-based code edits. We encourage the reader to watch the accompanying videos (in the non-anonymous Supplementary Materials). Together, the five examples required a total of 35 minutes of development time in Deuce (the videos are a few minutes longer because of pauses during the narration), resulting in approximately 100 lines of Little code for the final programs. At several points in the videos, we use the existing Sketch-n-Sketch ability to directly manipulate the size, position, and color of shapes in the output, as a way to indirectly update constant literals in the program. We use this feature for simplicity; the constant literals could, of course, be changed with ordinary text-edits.


In this example, our goal is to generate a target comprising concentric rings of alternating color. We start by writing a single red circle, using the expression (* 1 50) for the radius to anticipate that it will scale linearly for ring 2, 3, and so on. We use Abstract to extract a function, ring, that is parameterized over this index i, and we use Remove Arguments so that ring is parameterized over only i. We use Introduce Variable to give a name to the ring color, and then use text-editing to choose red or gray depending on the parity of index i. Next, we update the main expression with text-editing to map the function’s i over the list of indices (reverse (range 1 4)). We use the Abstract tool to extract a target function for drawing these concentric circles. We would like the target function to take cx, cy, ringWidth, and ringCount, but the first three of these are currently constants inside the ring function. To turn these constants into function arguments, we move the entire ring definition (and thus the constants of interest) inside of target, which then allows us to use the Deuce tools to introduce and rename the desired arguments.

Battery Icon.

In this example, our goal is to build a program that generates a battery icon, akin to those often found in operating system task bars. The design comprises three shapes: a polygon with rounded stroke for the body of the battery, a rectangle for the cap, and a rectangle for the battery juice inside. In addition to setting up the appropriate positional relationships, we want the color of the battery juice to change based on the amount that remains. Our development process from start to finish, which takes approximately 15 minutes (without narration), mixes text-edits and Deuce transformations throughout. Our general workflow is to incrementally add new shapes and features, often starting with hard-coded or copy-pasted expressions, and then iteratively repairing the program by adding new variables and relationships.

The first shape we add to the program is the polygon for the body outline. We use the Introduce Variable tool to give names to the top-left and top-right corners of the body, which are needed by subsequent expressions. We use the Make Equal tool to equate certain offsets among edges in the design, and we use the domain-specific Freeze tool to ensure that some of these offsets are always the constant zero. If we accidentally swap the usages of width and height variables, we can use the Swap Variable Names and Usages tool to correct the bug. The second shape we add to the program is the cap. Again, we use a combination of text-edits and structured editing tools, such as Introduce Variable and Move Definitions, during this process.

The last shape we add is the colored rectangle for the battery juice. After we add it to our list of shapes, we see that the juice appears on top of the body rather than behind it. We use the Reorder Items tool to move this rectangle earlier in this list, which results in the desired z-ordering. When prototyping, it is natural to define the width of this rectangle directly as a constant w, with the intention that it remains less than the width of the body. Later in the development process, we use text-editing to introduce a conditional expression that determines the color (i.e., green, black, orange, or red) based on the value of the percentage (/ w width). This expression appears in several guards of a multi-way conditional, so we use the Eliminate Common Subexpression tool to give it a name, which we Rename to juicePct. Now that the relationships are set up, we realize that it would be better to first define juicePct (the percentage denoted by a number between 0.0 and 1.0) and then derive w in terms of it. We use the Move Definition tool to drag the former above the latter, and Deuce proposes an option where the definitions are inverted, specifically, juicePct is redefined to be a constant percentage and w is defined in terms of it. We use the Add Range tool twice twice, once on w before it was rewritten and once on juicePct afterwards. In each case, the automatically chosen range was useful for allowing the slider to quickly manipulate reasonable values for each quantity.

At this point, our program generates the entire icon in terms of the design parameters. To finish, we turn the definition into a function using a similar series of Move Definitions and Abstract transformations as described at the end of § 2. This time, we realize that the Abstract tool did not make all of our desired constants into parameters. So, we use the Add Argument and Rename tools to reach our desired parameterization of topLeft, bodyWidth, bodyHeight, capWidth, capHeight, strokeWidth, and juicePct.

Coffee Mug.

In this example, our goal is to build a program that generates a coffee mug, in a way that it is easy to reposition and resize the logo. When developing the body of the mug and two concentric ellipses for the handle, the Introduce Variable and Move Definition tools help turn initially hard-coded shapes into the desired relationships.

When designing the steam, we use tools not already exercised in the previous examples. When we add the first steam puff, we use hard-coded constants for all of the points and control points of the path. This helps us get the initial design for the curvy puff, but makes it hard to move to a different position; all 12 constants must be updated by the same offset to translate it. We use the Rewrite as Offset tool several times to make the steam puff rigid. Then we use the Duplicate Definition tool to copy-paste (via Deuce rather than text-editing) the first steam definition twice. After changing just the initial position of each puff, our copy-pasted definitions contain nearly identical code. We use the Merge tool to abstract the three steam puffs over their differences (i.e. the position). We use Rewrite as Offset several more times to position the second and third steam puffs in terms of the first, and then again to position the first steam puff in terms of the mug; the effect is that the steam remains rigid and correctly positioned as the mug is translated to different positions. During these last steps, we move several definitions from the bottom of the program up to the top so that the related expressions are closer together; the Move Definition tool allows us to make such transformations without fear of breaking dependencies in or changing the binding structure of the program.

Mondrian Arch.

Inspired by the Mondrian programming-by-demonstration graphics editor (Lieberman, 1993), our final example is an arch, where two upright rectangles support a third horizontal rectangle, all of which are of equal width. As in the previous examples, we use tools like Introduce Variable often to help reorganize the code and text-edits to fill in arithmetic relationships. Unlike the previous examples, we use tools that manipulate concrete whitespace—Make Multi-Line to facilitate the step of going from one call to rect to multiple ones, and Make Multi-Line to make the arguments to these adjacent calls line up vertically, making it easier to distinguish the differences between all calls. These tools eliminate some of the tedious text-edits that arise when making such stylistic changes to the code.


In all of our examples, the Deuce transformations augment what is otherwise a natural, unrestricted text-editing workflow. This means that no matter what the limits of the provided transformations are, the user can always continue to work as in a plain text code editor.

5. Related Work

We describe several ideas in refactoring, structured editing, and other forms of interactive programming that are related to Deuce. Ko and Myers (2006), Lee et al. (2013), and Omar et al. (2017) provide more thorough introductions to the rich literature on these topics.

5.1. User Interfaces for Structured Editing

Perhaps the simplest way to expose automated tools for structured transformations is with a toolbar of menus. Several alternative user interface features have been proposed to integrate structured editing more seamlessly within the text-editing workflow.

Drag-and-Drop Refactoring.

Regarding the user interface challenges for identifying, invoking, and configuring refactorings (cf. § 1), Lee et al. (2013) propose DNDRefactoring, a tool that eliminates the use of menus altogether. They demonstrate how many common, traditional refactorings can be invoked unambiguously with direct manipulation interactions (specifically, drag-and-drop) without the need for any additional configuration. This is a compelling workflow for situations in which the user can (a) readily identify an intended refactoring based on a preconceived notion (e.g. its name), (b) unambiguously invoke the intended refactoring by a single drag-and-drop gesture, and (c) accept the default configuration of the refactoring. It would be useful to add drag-and-drop gestures to Deuce for transformations that satisfy these three conditions.

Our goal, however, is to provide a user interface for when one or more of these three conditions fails to hold. For identification, our menu displays potential transformations that the user may not already know by name (e.g. Make Equal) or may not realize is an option (e.g. inverting dependencies to Move Definitions safely). For invocation, our user interface allows gestures that require additional selection information (e.g. Make Equal takes an arbitrary number of source items and no explicit target, because it is automatically computed). Furthermore, our menu helps quickly choose between multiple potential transformations (and configuration options) triggered by the same user action (several transformations in Figure 5 are active based on the same selection state). Our approach—context-sensitive menus that appear near the selected elements—attempts to strike a middle-ground between the “heavyweight” menu-based approach in Eclipse and the no-menu approach of DNDRefactoring, providing structured transformations within the typical workflow of a text-editing environment.

Hybrid Editors.

Compared to “fully” structured editors, several hybrid editor approaches augment text-based programs with additional information to help with development, maintenance, and understanding. Barista (Ko and Myers, 2006) is a hybrid Java editor where structure views can be implemented to present alternate representations of structural items instead of text. For example, an arithmetic expression may be rendered with mathematical symbols instead of ASCII, a method may be accompanied by interactive documentation with input-output examples, and structures may be selectively collapsed, expanded, or zoomed. Omar et al. (2012) introduce a similar notion to structure views, called palettes, where custom displays can be incorporated based on the type of a subexpression. For example, a color palette can provide visual previews of different candidate color values, and a regular expression palette can show input-output examples for different candidate regular expressions. In Greenfoot (Brown et al., 2016), program text is separated into structural regions called frames, which are created and manipulated with text- and mouse-based operations that are orthogonal to the text-edits within a frame. Code Bubbles (Bragdon et al., 2010) allows text fragments to be organized into working sets, which are collections of code, documentation, and notes from multiple files that can be organized in a flexible way as needed for the task at hand. Outside of the views, palettes, frames, and working sets in the above hybrid editors, the user has access to normal text-editing tools.

Our approach is complementary to all of the above: in places where code fragments—regardless of their granularity and their relationship to alternative or additional pieces of information—are represented in plain text, we aim for a lightweight user interface to structurally manipulate it.

Refactoring with Synthesis.

In contrast to the direct manipulation user interfaces of DNDRefactoring and Deuce, Raychev et al. (2013) propose an appealing workflow where the user starts a refactoring with text-edits (i.e. showing an example of part of the result after the refactoring) and then asks the tool to search for (i.e. synthesize) a sequence of refactorings that complete the task. It seems likely that this text-based interface and the mouse-based interfaces of DNDRefactoring and Deuce would each be preferable for different tasks.

5.2. Refactorings and Other Structured Transformations

Automated support for refactoring (Griswold, 1991; Fowler, 1999; Roberts et al., 1997) has been aimed primarily at programs written in class-based, object-oriented languages, with relatively “coarse-grained” transformations (e.g. Extract Class and Extract Method work at the granularity of classes and methods, respectively). As we have described, our current implementation of Deuce is in the context of a functional language, and the set of transformations includes several “fine-grained” transformations that arise during various phases of the development process.

Refactoring for Functional Languages.

HaRe (Thompson and Li, 2013; Li, 2006; Brown, 2008) provides many transformations that are common when programming in functional languages, such as Haskell, where features including first-class functions (i.e. lambdas), local bindings, tuples, algebraic datatypes, and type polymorphism lead to tasks—such as Promote to Outer Scope, Demote to Inner Scope, and Generalize Expression—that are more “fine-grained” tasks than those supported in most typical refactoring tools for object-oriented programs. Like refactoring tools for object-oriented languages, HaRe employs a menu-based user interface (implemented in Emacs and Vim), which requires one of the named transformations to be explicitly selected from a menu and applied to the desired expressions in the code. Our user interface could be incorporated by HaRe to expose the supported transformations with lightweight, direct manipulation features.

HaRe provides a large catalog of transformations, many more than in our current implementation of Deuce. However, some of our transformations are, to the best of our knowledge, not found in existing tools. For example, HaRe provides Promote and Demote transformations for moving definitions into adjacent (outer and inner, respectively) expressions, whereas our Move Definition tools allow reordering of non-adjacent definitions, as well as dependency inversion. Also, the Make Equal tool is a non-semantics preserving transformation that we have found useful during the prototyping and repair phases of development. Lastly, we have implemented some domain-specific transformations useful in the setting of functional programming to generate SVGs.

Transformations for Program Development.

Reichenbach et al. (2009) identify the role of program metamorphosis, where intermediate steps of a transformation do not preserve behavior, or even where the overall transformation is intended to selectively deviate from the original. Steimann and von Pilgrim (2012) describe a tool for supporting refactorings that are specifically tailored to the particular program and task at hand and, thus, do not have well-established names. A goal of our work is to promote automated support for structured transformations throughout all phases of development, including ones which change program behavior and which may even be tailored to a specific domain or program.

While the majority of work on structured editors has focused on user interfaces and tools, relatively little has been developed about the meanings of programs that are undergoing transformations. Omar et al. (2017) define a calculus that serves as a foundation for reasoning about edits to an AST. A particularly interesting aspect of their work is their account for incomplete programs where holes have yet to be filled in. The programming process consists of iteratively transforming an incomplete program into a final one, so these foundations may help to inform the goals and design for the kinds of interactive editing environments that Deuce and many other structured editing tools aim to provide.

5.3. Direct Manipulation of Output in Sketch-n-Sketch

The user interface in Sketch-n-Sketch (Chugh et al., 2016; Hempel and Chugh, 2016) provides direct manipulation tools for (1) drawing shapes, (2) declaring intended relationships between shapes, (3) grouping and abstracting shapes, and (4) changing shape properties such as position, size, and colors. For each of these direct manipulation features, Sketch-n-Sketch automatically transforms the program text to match the user action.

Although most of the features provided in Deuce work for arbitrary Little programs (not just those used to compute SVGs), we have chosen to integrate our editor within Sketch-n-Sketch for two reasons. First, while the existing output-directed synthesis features in Sketch-n-Sketch attempt to generate program updates that are readable and which maintain stylistic choices in the existing code, the generated code often requires subsequent edits, e.g. to choose more meaningful names, to rearrange definitions, and to override choices made automatically by heuristics; Deuce provides an intuitive and efficient interface for performing such tasks. Furthermore, by allowing users to interactively manipulate both code and output, we provide another step towards the goal of direct manipulation programming systems identified by Chugh et al. (2016). These two capabilities—direct manipulation of code and output—are complementary.

6. Conclusion and Future Work

The primary motivation for our work is to demonstrate how a variety of structured transformations—not just refactorings—can be supported with automated tooling in a lightweight, text-editing workflow (cf. § 1). Our approach also aims to simplify the steps to identify, invoke, and configure transformations provided by refactoring tools (cf. § 5.1).

Murphy-Hill and Black (2008) define five desired characteristics that a floss refactoring tool ought to satisfy. We refine these criteria below according to our motivation, with changes marked in bold. Compared to the original criteria, we generalize the goals of such tools slightly in order to (a) support transformations beyond refactorings, and (b) provide a seamless interface for interacting with the user in situations where configuration cannot be avoided (e.g. to make choices about variable names and function boundaries).

  1. Choose the desired structured transformation quickly,

  2. Switch seamlessly between program editing and structured transformations,

  3. View and navigate the program code while using the tool,

  4. Avoid providing explicit configuration information, but do so seamlessly when needed, and

  5. Access all tools normally available in the development environment while using the structured editing tool.

We believe Deuce represents a proof-of-concept for how to achieve these goals for “floss structured editing.” We hope that our ideas will help inform future progress on making refactoring and structured editing a more common part of a typical programming workflow.

6.1. Future Work

We see several directions for subsequent work. One is to adapt and implement our approach for existing, full-featured programming languages and existing development environments. Another is to incorporate additional well-known transformations (e.g. (Fowler, 1999; Thompson and Li, 2013)). Independent of these choices about target language and transformations, there are several opportunities to improve on our methodology.

User Interface.

With respect to the selection of structural features, it would be useful to have a more sophisticated presentation than our bounding polygons, so that it is easier to select components among deeply nested structures. Furthermore, it would be useful to denote target positions by a more expansive notion of space between expressions rather than using only the single characters before and after items.

Additional Direct Manipulation Gestures

There are several opportunities to improve the user interface for invoking transformations. While our approach supports invoking transformations that require multiple selections, it would be useful to support drag-and-drop gestures (as in DNDRefactoring) for the subset of transformations that require a single source item and single target expression. For transformations that do not fit in the single-source single-target paradigm, it would be useful to perform user studies, following the approach of Lee et al. (2013), to identify intuitive mappings for more specific direct manipulation gestures than the selection- and menu-based approach in Deuce. For example, to invoke the Make Equal tool to copy expressions, two alternative user actions could be to (i) treat the first selection as the one to copy to the rest or (ii) select all of the expressions to be modified and then drag the expression to copy to one of them (which would copy the expression to all selections).

Code Diffs

To help explain each of the suggested transformations, our visual previews may be augmented to show code diffs, instead of just the transformed code as in our current implementation. It may also help to draw additional information on the source code relevant to a transformation—for example, arrows between source and target expressions, and between variable uses and definitions (as in DrRacket (—and to smoothly animate the code edits to further help explain the automated transformations.

Offscreen Widgets

It would be useful to provide information about selected widgets that are offscreen; these are currently not shown. Alternatives include information at the borders of the code editor to summarize the selection state of items in each direction, and a scrolling side bar view that depicts the entire program and relevant areas of selection state.


While our current approach displays which transformations are active given the selection state, it could further help discoverability to show what transformations may be active if additional items are selected. The challenge here is to display suggestions for selections and transformations that may be of interest without cluttering the display and overwhelming the user with too many choices.

Domain-Specific Language for Custom Transformations.

If a Little transformation can be implemented in terms of the active selections of a program, it can easily be plugged into our menu-based approach. To make the approach even more extensible, it would be useful to design a domain-specific language of rewrite rules and appropriate side conditions, so that Deuce may be instantiated with different transformations in different settings without having to implement each inside the tool.

Adaptive Concrete Syntax.

Our approach embraces plain text as a desirable format for programs, and, thus, the importance for programmers to make judicious choices about variable names, whitespace, and other formatting properties. Tools such as Move Definitions, Align Expressions, and Make Multi-Line provide a modicum of help when making these decisions, but there are still many factors that the user must consider entirely on her own—such as screen sizes, style conventions, changes in expressions that make them shorter or longer.

It may be interesting to consider how the editor could automatically convert between different plain text formats as these different criteria change. For example, maybe a particular multi-way definition is a good choice when, say, the maximum suggested line width for a given project is 80 characters but not when it is 60 characters. In addition to the transformations currently implemented in Deuce that help the user perform transformations on the abstract syntax of the program, transformations could help manipulate the concrete syntax as well.

Incremental, Structure-Aware Parsing.

Toggling between text- and mouse-based editing in Deuce is achieved with a quick keystroke, but the complete separation between modes has limitations. As soon there are any character changes to the text, the structure information (and any active transformations) become invalidated. It would be better to incorporate an incremental parsing approach (e.g. (Wagner and Graham, 1998) as used by Barista (Ko and Myers, 2006)) to tolerate some degree of differences in the source text, so that as much of the structure information as possible can persist even while the code has been changed with text-edits. This would help to further streamline, and augment, the support for structured editing within the full, unrestricted text-editing workflow.

The authors would like to thank Shan Lu and Elena Glassman for providing valuable comments that improved the presentation in this paper, Justin Lubin for contributing to the implementation of Sketch-n-Sketch, and the University of Chicago for awarding a Liew Family College Research Fellows Grant to help fund this work.


  • (1)
  • Altadmri et al. (2016) Amjad Altadmri, Michael Kölling, and Neil Christopher Charles Brown. 2016. The Cost of Syntax and How to Avoid It: Text versus Frame-Based Editing. In Computer Software and Applications Conference (COMPSAC).
  • Bragdon et al. (2010) Andrew Bragdon, Steven P. Reiss, Robert Zeleznik, Suman Karumuri, William Cheung, Joshua Kaplan, Christopher Coleman, Ferdi Adeputra, and Joseph J. LaViola, Jr. 2010. Code Bubbles: Rethinking the User Interface Paradigm of Integrated Development Environments. In International Conference on Software Engineering (ICSE).
  • Brown (2008) Christopher Brown. 2008. Tool Support for Refactoring Haskell Programs. Ph.D. Dissertation. University of Kent.
  • Brown et al. (2016) Neil Christopher Charles Brown, Amjad Altadmri, and Michael Kölling. 2016. Frame-Based Editing: Combining the Best of Blocks and Text Programming. In Conference on Learning and Teaching in Computing and Engineering (LaTiCE).
  • Chugh et al. (2016) Ravi Chugh, Brian Hempel, Mitchell Spradlin, and Jacob Albers. 2016. Programmatic and Direct Manipulation, Together at Last. In Conference on Programming Language Design and Implementation (PLDI).
  • Fowler (1999) Martin Fowler. 1999. Refactoring: Improving the Design of Existing Code. Addison-Wesley Longman Publishing Co., Inc.
  • Griswold (1991) William G. Griswold. 1991. Program Restructuring as an Aid to Software Maintenance. Ph.D. Dissertation. University of Washington.
  • Hempel and Chugh (2016) Brian Hempel and Ravi Chugh. 2016. Semi-Automated SVG Programming via Direct Manipulation. In Symposium on User Interface Software and Technology (UIST).
  • Ko et al. (2005) Andrew J. Ko, Htet Htet Aung, and Brad A. Myers. 2005. Design Requirements for More Flexible Structured Editors from a Study of Programmers’ Text Editing. In Human Factors in Computing Systems (CHI).
  • Ko and Myers (2006) Andrew J. Ko and Brad A. Myers. 2006. Barista: An Implementation Framework for Enabling New Tools, Interaction Techniques and Views in Code Editors. In Human Factors in Computing Systems (CHI).
  • Lee et al. (2013) Yun Young Lee, Nicholas Chen, and Ralph E. Johnson. 2013. Drag-and-Drop Refactoring: Intuitive and Efficient Program Transformation. In International Conference on Software Engineering (ICSE).
  • Li (2006) Huiqing Li. 2006. Refactoring Haskell Programs. Ph.D. Dissertation. University of Kent.
  • Lieberman (1993) Henry Lieberman. 1993. Mondrian: A Teachable Graphical Editor. In Watch What I Do: Programming by Demonstration. MIT Press.
  • Maloney et al. (2010) John Maloney, Mitchel Resnick, Natalie Rusk, Brian Silverman, and Evelyn Eastmond. 2010. The Scratch Programming Language and Environment. Transactions on Computing Education (TOCE) (2010).
  • Mealy et al. (2007) Erica Mealy, David Carrington, Paul Strooper, and Peta Wyeth. 2007. Improving Usability of Software Refactoring Tools. In Australian Software Engineering Conference (ASWEC).
  • Monig et al. (2015) Jens Monig, Yoshiki Ohshima, and John Maloney. 2015. Blocks at Your Fingertips: Blurring the Line Between Blocks and Text in GP. In IEEE Blocks and Beyond Workshop (BLOCKS AND BEYOND).
  • Murphy-Hill and Black (2008) Emerson Murphy-Hill and Andrew P. Black. 2008. Refactoring Tools: Fitness for Purpose. IEEE Software (2008).
  • Murphy-Hill et al. (2009) Emerson Murphy-Hill, Chris Parnin, and Andrew P. Black. 2009. How We Refactor, and How We Know It. In International Conference on Software Engineering (ICSE).
  • Omar et al. (2017) Cyrus Omar, Ian Voysey, Michael Hilton, Jonathan Aldrich, and Matthew A. Hammer. 2017. Hazelnut: A Bidirectionally Typed Structure Editor Calculus. In ACM SIGPLAN Symposium on Principles of Programming Languages (POPL).
  • Omar et al. (2017) Cyrus Omar, Ian Voysey, Michael Hilton, Joshua Sunshine, Claire Le Goues, Jonathan Aldrich, and Matthew A. Hammer. 2017. Toward Semantic Foundations for Program Editors. In Summit on Advances in Programming Languages (SNAPL).
  • Omar et al. (2012) Cyrus Omar, YoungSeok Yoon, Thomas D. LaToza, and Brad A. Myers. 2012. Active Code Completion. In International Conference on Software Engineering (ICSE).
  • Raychev et al. (2013) Veselin Raychev, Max Schäfer, Manu Sridharan, and Martin Vechev. 2013. Refactoring with Synthesis. In Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA).
  • Reichenbach et al. (2009) Christoph Reichenbach, Devin Coughlin, and Amer Diwan. 2009. Program Metamorphosis. In European Conference on Object-Oriented Programming (ECOOP).
  • Resnick et al. (2009) Mitchel Resnick, John Maloney, Andrés Monroy-Hernández, Natalie Rusk, Evelyn Eastmond, Karen Brennan, Amon Millner, Eric Rosenbaum, Jay Silver, Brian Silverman, and Yasmin Kafai. 2009. Scratch: Programming for All. Communications of the ACM (CACM) (2009).
  • Roberts et al. (1997) Don Roberts, John Brant, and Ralph Johnson. 1997. A Refactoring Tool for Smalltalk. Theory and Practice of Object Systems (1997).
  • Steimann and von Pilgrim (2012) Friedrich Steimann and Jens von Pilgrim. 2012. Refactorings Without Names. In Conference on Automated Software Engineering (ASE).
  • Teitelbaum and Reps (1981) Tim Teitelbaum and Thomas Reps. 1981. The Cornell Program Synthesizer: A Syntax-Directed Programming Environment. CACM (1981).
  • Thompson and Li (2013) Simon Thompson and Huiqing Li. 2013. Refactoring Tools for Functional Languages. Journal of Functional Programming (2013).
  • Tillmann et al. (2012) Nikolai Tillmann, Michal Moskal, Jonathan de Halleux, Manuel Fahndrich, and Sebastian Burckhardt. 2012. TouchDevelop: App Development on Mobile Devices. In International Symposium on the Foundations of Software Engineering (FSE).
  • Vakilian et al. (2012) Mohsen Vakilian, Nicholas Chen, Stas Negara, Balaji Ambresh Rajkumar, Brian P. Bailey, and Ralph E. Johnson. 2012. Use, Disuse, and Misuse of Automated Refactorings. In International Conference on Software Engineering (ICSE).
  • Wagner and Graham (1998) Tim A. Wagner and Susan L. Graham. 1998. Efficient and Flexible Incremental Parsing. ACM Transactions on Programming Languages and Systems (TOPLAS) (1998).

Appendix A Examples

One example was presented in Figure 3; the rest are below.

Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description