kniterate notes 5

This is the fifth in a series of blog posts about the Material Programming Project. We are developing malleable knitting software for the Kniterate, a semi-industrial knitting machine. This post is a bunch of notes from the past few weeks. The first post, on the Knitout project, is available here.

It’s been a few weeks since I’ve had a chance to write up notes on Material Programming, mostly because a lot of different things have been happening. Some of them I’ve written about here:

I finished the first draft of the Kniterate-Knitout visualiser →
My friend Jake presented his PhD thesis on modular open-source manufacturing →
Ink & Switch came to visit both Chelsea’s Smart Textiles Lab and the CCI →

I also made a bunch of updates to the Material Programming site, which now gives a better idea of our current work and direction.

Kniterate-Knitout Visualiser

the visualiser with waste section selected

In preparation for the second material programming workshop (which will take place at the start of May), I finished the first version of the forked CMU visualiser, adapted for the Kniterate machine. (here’s the original). I also wrote a cheatsheet guide to the tool and a couple of example scripts.

The main changes I made were to add a KCode export button (very easy!) and to add an automatic waste-section addition (more difficult!). In both cases, I was adding existing scripts from Gabrielle Ohlson’s knitout-backend-kniterate repository into the visualiser.

What was great about all of this is that it was possible to do everything client-side. After some tidying of scripts in the repository, I added in the knitout-to-kcode script almost verbatim, calling it after the JS parsing step to export straight to the KCode format that the Kniterate takes.

Reworking the Waste Section

The waste section was more challenging, as I wanted to use something more similar to the kniterate editor’s waste section, and had also been encountering a bunch of bugs in using the original script, something I talked about in the last post.

Testing integrating multiple working yarns in the waste section

I decided to significantly refactor the code, adding in some functions to keep track of carrier state, to avoid the issue where a row would be brought in in the wrong direction. I wanted the waste section to be largely invariant to the cast-on direction of the eventual piece, and also to be able to integrate an arbitrary number of yarns that would be brought into work further up in the sample – I spent a bit of time integrating these in the waste section so they’d always end up on the correct side.

I was pretty pleased with the results in the end, it’s satisfying to click the waste section box and see all the relevant stitches appear, and I really understand how the code works now. This is a bit of a placeholder though – ideally I’d like to make a fork of the knitout-frontend-js parser that abstracts stuff like waste sections and bindoffs into callable functions, rather than relying on buttons in the interface (after all, the button isn’t very malleable).

Racking

Knitting every other front and back bed needle on 0 racking (needles directly face each other) for a 1x1 rib (top) vs 0.5 racking (‘half pitch’) to knit both beds

One minor but interesting issue I came across was a mismatch in assumptions about how racking works between the Shima and Kniterate machines. Racking is an operation where the front bed is moved relative to the back bed. In normal operation, needles sit directly opposite one another (racking 0), but if you want to knit with same-numbered needles on both beds in the same row (e.g. for the ‘both beds’ row during cast-on) then they need to be offset by some fraction of a stitch to avoid hitting each other. Racking by more than one stitch can also be used to create cabling effects, where knitted stitches on the front are moved relative to the stitches on the back bed.

The underlying validation in the JS->Knitout code wants the rack offset to be either an integer number of stitches, or some integer+0.25 – my vague theory is that Shimas use .25 increments because they can have 4 beds, though I don’t know if that’s true. The Knitout->Kniterate validator wants you to rack by 0.5 stitches, which is also what the Kniterate machine likes. It’s a minor point but so interesting to encounter the way that certain machines can end up ‘baked in’ to the software in really subtle ways.

Knitscript first encounters

As I was working on the waste section code, I was starting to draw up a list of things I thought were missing from knitout-js, namely:

a representation of the machine ‘state’, in particular the position of the carriers
a way of writing higher-level commands, like “4 rows of interlock” “2x2 rib” without needing to specify everything at the stitch level
some kind of abstract ‘model’ of the machine, so things like the desired racking can be expressed as part of a driver or config file

I’d also had the Knitscript paper saved for a while, and excitingly, this is exactly what it’s about. Knitscript also came out of CMU – Megan Hoffman and Lea Albaugh are the two lead authors. Knitscript introduces a bunch of new abstractions on top of knitout, managing the state of the knit machine as an extension of the Python interpreter’s state. It works by executing instructions on a virtual model of the machine, which also has a model of what makes a ‘stable’ piece of knitted fabric.

I found that I couldn’t get the code working out of the box, and got the sense that it had been refactored, but without the instructions changing. The fixes weren’t that exciting – if you wanted to replicate them, I documented the changes in an issue and made a pull request with some updates to the initial instructions.

Just from reading the paper, they do seem to make a number of assumptions that mean that this won’t work out of the box on Kniterate machines – such as automatic carrier handling using the inhook and outhook Knitout commands, which don’t translate into knitout operations. Writing a kniterate driver for this might be very interesting!

The End of GCode

gcode memes :)

Jake is an old friend from the Media Lab, and it was really lovely and exciting to watch his PhD defence at the start of April. His PhD was in the Center for Bits and Atoms, and explored the use of modules and models to represent the state of machines, as the basis for recasting machine control as a constrained optimisation task.

It was really cool hearing him talk in detail about the material concerns of creating abstract models of machining processes. Sometimes I can feel like people are really excited about the idea of creating a new paradigm for doing hard things (like making abstract representations of machines) and then much less enamoured with the inevitable translation work involved in adapting these representations to actually work in reality (I feel similarly every time someone claims to have solved schema-translation problems!). That interface between physics and standards is where so many cool things happen.

The presentation focussed on two examples of integrating feedback between machine state and instructions:

optimising endmill temperature and speed in CNC machining
optimising extrusion stability in a 3D printer

In both cases, Jake uses constraint solvers to optimise the process a particular material and part, at the fastest rate that will keep the machine happy. The 3D printing example was particularly funky as it involved a bunch of rheology to understand the forces in the extruder, modelling the behaviour of molten materials. The feedback idea was particularly appealing in this context – instead of needing to produce a number of samples to calibrate each time, a more general model of the material could be used.

One of the most interesting parts was how he talked about how this could generalise, and also why this was a particularly interesting thing to be working on now. Lots of the optimisation work was based on Jax, which allows the definition of large and arbitrary cost functions. The advent of LLMs has also led to a remarkable increase in the power and availability of general-purpose optimisation tools like Autograd, and just-in-time compilation.

What was also super impressive about this project was how many of the tools used were made from scratch – like the retrofittable clank! machine controllers, and the OSAP ‘Osap Systems Assembly Protocol’, which allows arbitrary pieces of modular hardware to be networked and integrated together as a single system.

neat OSAP protocol diagram

There were also a bunch of really nice concepts I’d not come across before. One was out-of-band insrumentation – the use of measurements of proxies for what’s happening when you can’t measure something directly, like listening to the sounds a machine makes to make inferences about how the motors are doing. It was also really interesting hearing about which processes weren’t amenable to this kind of feedback – like scaled-up semiconductor manufacturing, where repeatable and extremely fast feed-forward processes take precedence.

When asked at the end what determines whether something is easier or harder to represent in this manner, the response was that “it depends on how differentiable the process is”. It made me think about how this kind of work might apply to knit, especially in working out things like tension control for smaller-batch processes. It’s cool to see tools designed for the median user (who is making a small amount of something), rather than the most expensive one.

Jake we love you! Super cool!

Ink & Switch Visit

After we spoke at the Ink & Switch meetup last December, Peter van Hardenberg asked if he could come return the favour and talk at the CCI while he was in London in April. We also arranged a visit for a few of his collaborators at ARIA to come and see the Smart Textiles Lab in Chelsea, and showed them the Kniterate and some of the projects we’d been working on.

During Peter’s Chelsea visit, we talked a lot about what it means to abstractly represent machines – how to do this well, and also how to make use of mathematical abstractions like lenses as representational frames. He also used their new tool Patchwork to create a fork of the knitout visualiser, which was a really exciting way of dealing with client-side code.

In the talk Peter gave at the CCI the next day, he talked a lot about the tools they work on at I&S, and their philosophy toward toolmaking more generally. I’ve been a big fan of Ink & Switch’s work since encountering their essay Cambria while working on schema design at Knowledge Futures back in 2020, and it’s been really exciting to see how their work – and the way they articulate their work – has developed over the past few years.

There were some really lovely parts, including Peter’s articulation of the average computer being ‘less a bicycle for the mind, more of an aircraft carrier’, and totally unsuitable for the needs of the average user. He also encouraged everyone present to seek out ‘narrow waists’ to have as a compilation target for projects – text files being a great example.

I also really enjoyed the line: “you are in a contested space whenever you use technology”. I think a lot about how wasteful contemporary technology is, and how oriented around profit motives rather than social value, even in really basic assumptions. Questioning these or trying to think otherwise can take up loads of time, and it’s not always clear at the outset whether you’re just being stubborn, or actually have a point. I suppose the important thing is to insist on what you think is right.

Afterward I felt a renewed enthusiasm for pushing the limits of the knitout visualiser interface rather than immediately pushing on to something ‘more serious’.

further thoughts

I linked these things together because to me they feel very related – I left Jake’s talk feeling very inspired by some of the possibilities that exist now to do old things but in interesting ways, and for how exciting things like manufacturing can be when you think about how to optimise for small and open-source use. Similarly I’m feeling a renewed enthusiasm for the browser, and client-based sites. I can get very bogged down in the limitations of these things sometimes, and it’s nice to have that refreshed.

While I’m excited to play with Knitscript, hearing Peter’s talk has given me a renewed enthusiasm for spending a bit more time with browser-based knit tools and the idea of doing things in the client more generally and also push it a bit. I want to try some async code, include fetch requests, just find it’s limits a bit before pushing on to the next thing.

That said, I do want to spend some time with Knitscript and get a better sense for how it’s put together, I think it does a lot of things that I’d really enjoy. I’m also really excited for the first workshop, and to see what the students do with the work. I’m taking a lot of inspiration from a course my friend Dan teaches called Metatool, aimed at architectural computation students making their own architectural tools.