Revisiting "AI"

 

Picture from here.

 

I discussed a while back my opinion on the stampede to bring AI into the kitchen, living room, workplace, and bedroom that seems to be going on. 

 

My opinion isn’t all that different though I’m now a bit more informed. I believed I referred to ChatGPT (and others) as a stochastic parrot—that is, a device that is able to reliably generate convincing language but does not understand the meaning of the language it is processing. 

 

ChatGPT is not, exactly, such a device. It is a language prediction mechanism in the sense that given its broad training it can respond to a statement (or “prompt”) in a manner consistent with how its model of the language suggests is the desired response. This is quite a bit more sophisticated than a stochastic parrot. I am not saying large language models (LLMs) are conscious or have any understanding of the language they generate in the human sense of that word. I am saying they are extremely adept and sophisticated in mapping how language works.

 

This is not language at the sentence or word level—though ChatGPT maps that, too—but a much broader and deeper map.

 

Consider a regular visual map—like what we used to use on road trips when I was a child. It’s a piece of paper with coordinates on it and imagery. If you have the right x/y (east/west, north/south) coordinates, you can find a city. That’s a two-dimensional map. You can find a three-dimensional map of the solar system here. However, since you have to tell the map when you’re looking at the solar system, it is, in effect, a four-dimensional map. With those four coordinates, you can place a planet geographically.

 

However, to go forward with that metaphor, each of the planets influence the other and they’re all influenced by the sun. Light comes from the sun, for example. If we added that to the map, that might come in as a fifth dimension—a separate coordinate. You can see how this gets complex pretty quickly. While computers can work with multidimensional math easily, humans have more trouble. We want to actually understand such things—and we’re pretty much embedded in a four-dimensional brain. 

 

Physicists came up with the idea of phase space. This is a space where all physical quantities are represented as different dimensions and a given state is described as a coordinate set in those dimensions. It’s not hard to see the utility of such a description—and it’s not much of a leap that something like that level of complexity would be needed to describe all the intricacies of language. That’s not all of what goes on inside of ChatGPT but a complex multi-dimensional representation of language is part of it.

 

There’s a good description how LLMs work here. I’m not going to reproduce that. Timothy Lee and Sean Trott, the authors, did a much better job than I ever could. So, you should go read it. Wikipedia has a broad discussion of LLMs here.

 

In place of what I described as phase space, LLMs use a word vector. Word vectors mathematically represent a word in all its dimensions. Like the coordinate systems I was describing before, and like phase space, you can determine something approximating distance between words by referring to them within a complex coordinate system. If you look at Washington, DC, and Baltimore, MD, and get their x/y coordinates, it’s not a difficult mathematical operation to determine that their close to one another. Similarly, if you look at Baltimore and Boston, MA, you can tell Boston is farther away from Baltimore than Washington. Seattle is farther away yet. 

 

Using word vectors, you can develop maps of words that are close to one another in use. Or far away. Or build nets describing sets of words. Computers don’t balk at numbers of dimensions—LLMs use hundreds or thousands. GPT-3 uses over twelve thousand. I suspect the number of dimensions will only increase as more computing power is applied to training.

 

Ah, yes. Training.

 

One of the interesting things about LLMs is that they are not programmed per se. They analyze reams of material on their own—the training. Again, Lee and Trott discuss this in digestible detail. I’m not going to go into it here.

 

The next level of LLMs is the emergence of word predictions from the raw word vectors. Language has syntax and grammar—it is organized into meaningful packets. So is computer language code. When the LLM is trained on a language, that underlying structure is embedded in the dimensional relationships between the word vectors. LLM processing takes input and applies it to transformers. The input to a transformer results in a more meaningful result. That result can be further input into another transformer. The final result set is the predicted output from the input prompt. (Again, refer to Lee and Trott.)

 

It should come as no surprise to anyone that a powerful engine such as an LLM is trained on a great deal of language input that it would return sophisticated language as a result.

 

But there’s more to this—and this is the important bit.

 

GPT-3 recently (reported in July, 2023) that it had scored highly in tests of reasoning by analogy. Reasoning by analogy is something SAT and other college entry exams test. You’ve seen this. The test presents evidence (Given a letter sequence, abcde, eabcd, deabc, what is the rule? Or, what is the next entry in the sequence? That’s reasoning by analogy.)

 

Now, it’s interesting that GPT-3 failed in extracting the analogy from text but it did extract it from prompts. Is this evidence of analogic reasoning?

 

For my part—and the article I linked to above suggests the same—I think it indicates that reason by analogy is embedded in the language training of the LLM. By this, I’m suggesting that what we are seeing is not intelligence in the LLM but intelligence embedded in the training material as represented by the LLM.

 

I think this is borne out by the LLMs being used to determine protein folding. A team at Meta thought that though the “L” in LLMs was intended for language, it really just signifies a packet of data that can be analyzed. Reconfiguring the LLM to use protein chemistry as training input instead of language seemed a productive path.

 

It is, of course, far more complex than I have stated. My point is that the LLM produced plausible protein sequences similar to the way it produced plausible language constructs. Let’s remember that the capacity of the LLM to retain data is far greater than any human and its ability to analyze multi-dimensional space is far superior. By representing data as digestible material to the LLM, the inherent intelligence imbued in protein systems that are the product of a billion years of evolution is amenable to analysis. 

 

This does not mean the LLM understands protein chemistry anymore than it understands language—in the human sense. What it does analyze the underlying structure beneath the training material and, given a catalytic prompt, presents plausible combinations that conform to that underlying structure.

 

Given this point of view, LLMs become more interesting. They are not conscious. They do not comprehend what they are doing. But they are potentially powerful tools  to express the underlying sophistication and intelligence of the material they’re given.

 

That lack of comprehension shows up regularly—anyone using an LLM without oversight is just asking for trouble. AtomicBlender—a YouTube channel I follow—asked ChatGPT to design a nuclear reactor of the future and it gave surprising—if impractical—results.

 

I think that is going to be the main problem of using LLMs: the output must be verified. In the case of protein chemistry, a solution candidate would have to be verified experimentally and be assured it didn’t violate existing patents. LLM produced original material cannot be taken at face value.

 

LLMs are a powerful tool. The current situation reminds me of something from the Larry Niven, Protector: Intelligence is a tool that is not always used intelligently.

 

State of the Farm: August, 2023

 

This will not be a long post as I’m in Pennsylvania for a family reunion.

 

I had planned an interesting post but left without the requisite materials. So, we’ll work with what we have.

 

The end of July and beginning of August is the beginning of harvest season. The first incoming is blueberries—of which we got an enormous amount, dwarfing all previous years. This is due to two reasons: 1) our blueberry bushes have finally matured, and 2) we’re using bird netting effectively.

 

We tried to use bird netting last year but failed miserably. Putting up the netting ended up destroying our harvest. I just couldn’t get the netting over the bush without ripping off the ripening blueberries. This year, I built two wooden frames over the low bush and high bush blueberry in the main garden. In addition, we had planted a large space area on the west side. 

 

This year we used cattle fence panels (Remember those? They’re in the picture above.) to make a large space: about 16x18 feet. This is interesting since the cattle panels only come in 16ftx50in units. We needed four sections so cut two 16x50 panels in half and wired them to four 16x50 sections. We staked them into the ground and it gave us a very large area. This is wide enough and high enough that we can let the high bush blueberries grow quite large without ever having to trim them much. 

 

That said, we still put bird netting over all the frames and that was an incredible PITA. The bird netting we used was the normal ½ inch mesh that we’ve seen. It’s always difficult, catches everything, tangles on itself and any leaf/branch/rock/shoe it comes in contact with. We did it but only managed by swearing the air blue.

 

After this, I looked on line. Surely, somebody had to do this better than we did. It turns out we used the wrong netting. While the thin variety is used quite a bit, when you look at what professional growers use, it’s a braided material that appears much less difficult to handler. Several videos I saw had the growers drape them right over the bushes themselves—which we will not be doing. In addition, when we were driving up in Vermont recently, we saw that some blueberry farmers were hanging long streamers of Mylar. My guess is this was to repel the birds.

 

So far we’ve gotten a few gallons of berries—it’s easier to measure it in pounds. Say 10+ pounds. The last harvest isn’t in yet but after that we’ll (Retch. Groan. Yowl.) remove the netting.

After the blueberries, come the pears and the Aronias. We’re doing the pears a little early this year. There’s been so much rain that some of the fruit is splitting. We could leave it on the tree but it’s a risk. Taking them from the tree, we’ll wait a week or so and they’ll ripen on the table. Not as great as letting them ripen on the tree but still quite good. Plus, we don’t have to fight the wasps for them.

 

Aronias are an... acquired taste. They always taste dry in my mouth even though they can be quite juicy. It comes from the astringency. I planted two trees with the idea I’d plant more if I liked them. I don’t much but the tree is doing well and it’s a shame to waste the fruit. I think I’ll press the fruit and see if I can make the juice palatable.

 

One new tree that’s coming on line is the mountain ash. Its berries should be ripe when we get back. We planted the tree because the paw paw we planted near the same spot wasn’t doing well. Now, both the ash and the paw paw are doing well. We’re still undecided about what to do next. Lose the ash? Let them grow together? We really like the paw paws.

 

We have five apples we’re trying to harvest. (We have some crab apples but they hardly count.) Two of them are still too young. One we planted a while back and lost the label so we’re not sure what it is. It seems to have a cinnamon flavor. The remaining two are the Granny Smith and the Sops of Wine. 

 

The Sops is a terrific apple. Not too sweet and with a spicy bite. It is coming in like gangbusters. I had to put props under the branches before we left The Granny—well, I’ve been fighting that Granny for twenty-five years. It never gives much of a harvest. We’re talking to a tree service to see if they can prune it right and help us get it under control. If not... well, Applewood is nice to work with.

 

The Cornelian Cherries are just about ready to pick. I’m going to have to make room in the freezer or make wine of them instantly. We’ll see which. 

 

We’ve reached the point in the main garden where the day to day work is less. The plants are just so well entrenched the weeds are being out competed. The biggest job I have is to make sure the melons don’t eat the bok choy or the pole beans and the cucumbers don’t eat the pintos. Work waiting for me when I get back.

 

As I said last time, the beans haven’t been setting much in the way of fruit because of the heat. That’s lessened some so I’m hoping we’ll see more. We started pulling in eggplant. The lettuce is in a shady spot so we’ve had good salads. Other crops such as sugar beets, cabbages, bush beans, and the like, are doing well. We’ll be harvesting them, soon, too. 

 

I’m a little nervous about the potatoes. They’re starting to turn yellow. Wendy thinks this is when they die back. I think it’s a bit early. Still, it has been a hot summer. We pulled some the other day and the tubers look good. 

 

All said, the harvest looks promising.

 

Arts_&_Crafts_IV: Ball Boy and Chocolate Man

Pottery: It’s time to be ambitious. 

(Ball Boy is an author photograph a piece of meso-American pottery from the Harvard Peabody Museum. Chocolate Man is from here.)

 

I’ve been interested in meso-American pottery for the entire last year. I’ve tried various things (see here.) with some success. I decided to actually attempt to replicate some forms.

 

Ball Boy is a container I found in the Harvard Peabody collection. I’m sad to say I don’t have the identifying information from the collection. Chocolate Man is a Mayan cacao container. Both are pre-colonial.

 

Let’s start with Ball Boy.

 

I actually wanted to get fairly close to Ball Boy’s actual appearance, including the painting. I decided to hand build him by using a puki as described by Andy Ward. Pukis are essentially molds that pottery is pressed into. This particular puki was a bowl that was roughly spherical shaped. Then, once I had the shape made and properly smoothed, I popped that one out and made the second one. Now, I had two halves of a ball. I “glued” them together with slip, clay wet to the point that it’s more liquid than solid. Once that was done, I tapped it into a spherical shape.

 

Then, I made the features and used slip, again, to glue them into place.

 

 

By the time I was done, I had to show him upside down since there was a hole at the top where he rested. I didn’t want to flatten the bottom.

 

 

Then, I put his legs on.

 

Now, I had to paint him. 

 

It’s already clear that I had made a few mistakes on the face. The biggest one was making the ball too small. The original Ball Boy was quite large—basketball sized. This ball was about the size of basketball, too—if it was a basketball for toddlers. This meant that the size of the features were too large in proportion to the ball. I decided to continue with it.

 

Ball boy has some interesting features. First, was his color. I mixed up some underglaze to what I thought would be the right color. A lot of times, the underglaze shows its true colors before firing. Not so much this time. 

 

What came out of bisque firing was interesting but not quite right. Bisque firing is low temperature. It’s enough to turn clay into ceramic but it’s not chemically complete for what we do at the Arts Center. It makes the pot strong enough to hold conveniently, paint, etc. 

 

It’s not always clear what the colors are going to look like until the glaze firing. The temperature is considerably hotter. Glazes take on a glass like consistency. True color comes out. What goes into a glaze firing—

 

 

—isn’t necessarily what comes out.

 

 

I had a different goal for Chocolate Man. At this point, I can’t hope to equal the amazing painting. But I did want to approximate the form.

 

Again, I used a puki. This time, it was a bucket. When you look at the form of the original Chocolate man, it’s essentially a bucket with legs and a top. Once I pulled it out of the puki, I had to smooth it and put handles and feet on it.

 

The top I turned on the wheel. It was just easier.

 

While I didn’t want the original paint job, Chocolate Man needed some kind of decoration.

 

I’ve been reading a lot about the Nazca Lines lately. These are geoglyphs: human made images using the earth as canvas. The Nazca Lines are particularly interesting. They are elaborate designs of animals, people, or plants made by removing the topsoil over intricate paths. 

 

I used a DIY decal technique I read about. I traced out the image I wanted and scanned it. Then, I reduced it to the size I wanted and printed it out on newsprint. Then, I took the image to the studio and carefully “inked” over it with red underglaze—I’ve been enjoying using the red underglaze for a while now. Once that was done, I let it dry. Then, I wet the surface of the unfired Chocolate Man and carefully sponged the image onto the target pot. The whole process is described here.

 

The nice thing about this technique is that you do it all on greenware. Then, it can be repaired or enhanced by working over the image at the bisque stage. Again, you never know what you’re going to get at the glaze stage.

 

 

 

 

 

 

 

 

 

 

But it came out all right.

 

 

 

 

 

 

 

 

 

 

 

 

Obviously, none of these are perfect. I could have done much better handles on Chocolate Man. The top doesn’t have the same conical shape as the original. Pretty much the original is better than the copy in every way.

 

I’m not happy with the eyes on Ball Boy. Ball Boy came out as more of a parody of the original than anything else. He’s laughing at me. I can tell.

 

Still. I’m getting there slowly.

 

 

State of the Farm: July, 2023

 We’ll start with the bad news. All of our chickens have been killed. 

 

This happened a little over a month ago but I haven’t brought it up. It takes a little time to process this scale of loss. This is not an emotional loss like that of a dog or cat—these chickens are livestock: egg layers with meat at the end. We treat them well but they are not pets.

 

We had 5 adult chickens. These were our main egg layers. We were considering “retiring” them—meaning, turn to meat—in the fall. But we hadn’t fully committed to that decision. We had 2 Delawares and 3 Bielefelders. The Delawares were great. Good layers all year round except when molting. The Bielefelders were more problematic. They weren’t great layers and they had a tendency to be “broody”—hens that would sit on eggs trying to hatch them. We had already tried letting one of those broody hens actually hatch a collection and that failed.

 

In addition to the 5 adult chickens, we had 8 young chickens ranging from six to eight inches in height. These were to be the replacement chickens. The plan was to retire the Bielefelders—they really weren’t producing and, maybe, try to get the adult Delawares and the young Delawares to live together.  The last idea was tricky as chickens have a tendency to peck intruding chickens to death.

 

We had moved the adult chickens to our auxiliary coop and put the young chickens in the main house. 

 

So, it’s early June and my wife comes in quite upset: all of the chickens are missing or dead. There is one largish chicken corpse left in the main coop and two bodies left in the auxiliary coop.

 

We’d had a major incursion about ten years ago when a bobcat tore apart the PVC framed coop I’d built. At that point we’d had four survivors: two in the PVC coop and two that had been safe in auxiliary coop. I had torn down the PVC coop and built Stalag-17, that main coop, and reinforced the auxiliary coop. I figured I’d learned my lesson: when you build something resembling a farm, to nature it looks like a great buffet. We’d already refenced the main garden and topped it with an electric wire to keep out deer on the top and rabbits on the bottom. 

 

I even reinforced the main coop with hardware cloth to keep out mice and such. 

 

Upon investigation, we found one corner of the main coop that I hadn’t covered with hardware cloth. There we found scraps of feathers. From the evidence, we concluded we’d been attacked by a family of weasels: the right size and the right behavior. The largest youngling just wouldn’t fit through the fencing.

 

The auxiliary was a bit tougher. The weasel scenario didn’t seem to apply since these chickens were ten to twelve pounds: much too large for the fence gaps. I finally found a corner where some of the old chicken wire had rotted away and the predator had enlarged into a hole. 

 

We’ve had chickens for close to thirty years. As I’ve mentioned, this isn’t the first-time nature came calling. But this time is different.

 

Our son moved back in to the house last year and stayed with us. Up to that point, we’d been having a daily breakfast of eggs. For the two of us, this almost exactly matched the chicken egg output. But he can’t eat dairy and, to me, eggs without cheese are not worth eating. On top of that, my wife went back into the office three times a week after the first of the year. Commute times being what they were, this precluded breakfast of any kind, much less eggs. The eggs stacked up.

 

My son moved out in the spring and we started making a dent in the eggs. We also made some egg bread and other egg-based treats. But we were still producing more eggs than we were consuming. 

 

Now, we have no eggs coming in at all.

 

The chickens aren’t a lot of work but they are work. They produce a material we don’t eat as much as we used to. There’s a certain minimum number of hens we need for them to survive the winter and that minimum produces more eggs than we consume. Added to that, the chief impediment for us to travel is the chickens. The greenhouse only takes a minimum of effort in the winter. The garden is only active during the summer. 

 

Over the last few years, the maximum time we can leave the homestead is a week. While this is fine for some locations, it doesn’t make any sense to travel all the way to, say, Greece with that constraint: 1 day travel there, ½ day to recover, 1 day travel back, to get 4.5 days of visit. When we visited China for nearly three weeks, it took a great deal of planning to get someone to take care of things while we were gone.

 

So, we’ve been discussing whether we want chickens at all. That’s going to be a discussion over the next year and we’ll make a decision next spring.

 

But the rest of the garden is doing very well.

 

The picture above shows the main garden: cabbage, cold weather crops, beans, melons, tomatoes, basil, peas, turnips, and sugar beets. The Japanese beetles  have made a come back this year. So far, they haven’t eaten everything down to the bone like they did previous years. But it’s been raining so much I haven’t been able to spray properly. 

 

While the whole peach family didn’t blossom this year, we did get blossoms on paw-paws, apples, quinces, pears, and grapes. Then, a strange little freeze came through in April and burnt the growing plants selectively—one part of a tree had frostburn and another part of the tree would be fine. All of this was near the main road. 

 

The main house never registered a temperature for an actual freeze but we’re about thirty feet higher than the road and the other side of the road is lower still. My hypothesis is that area got quite cold at the road and below but that didn’t get up to the trees. Then, a truck drove by or a light, morning breeze and strands of freezing air wound over the lower property.

 

Anyway, looks like we lost the paw-paws.

 

The June rains were warmer this year and the fruit trees and grapes loved that. Then, warm rains came and everything liked that. We’re on a hill so the water didn’t pool anywhere and the slope isn’t so great that we had washout. We packed the last of this year’s chicken manure around various plants and were richly rewarded. The grapes that have been low producing the last couple of years told me that this is what they were waiting for.

 

And the potatoes are huge. We planted about twenty pounds of seed potatoes and we’re hoping for a five-fold return. That’s the hope anyway.

 

There are some funguses that I need to watch. Several raised beds have strong, and poisonous, mushroom crops and there is some kind of brown wilt on the grapes. Some spraying is required and I’ll do that as soon as we get a string of sunny days. There are some beds that look like they have a nitrogen shortage so more manure needs to be distributed. We were going to plant some pumpkins but the stores didn’t have varieties we wanted so that moment has passed.

 

So, aside from the chickens, it’s been good so far.