Bartek Śliwa: So, going back to your railcar example, if we have a certain set of data that we observe based on camera footage, those neural networks know that this is a specific railcar with a specific number, and they can tell us whether that’s what we were expecting.

Bartłomiej Gładys: And this is where the learning process comes in. We train a neural network by giving it images. For example, with the network we used in that project, we would provide images and then assign what we call labels to them. So, for instance, we would mark where a railcar number appears in the image. Then the learning process begins, where we “penalize” the network, so to speak, by adjusting its weights every time it makes a mistake. Once it correctly identifies the location of the number and the number itself, we stop making those adjustments. At that point, we assume that the network has learned it correctly. Another important thing is that it needs to be able to generalize to other data it hasn’t seen before. And that’s also one of the difficult parts of machine learning: choosing a training dataset that will work across a wide variety of scenarios. For example, what happens when different railcars appear, when the weather is bad, when something happens to the camera, or when the camera is tilted at a different angle? There are many situations like that, which means we have to keep expanding our training datasets.

Bartek Śliwa: Mm-hmm. That sounds like a fairly large, well-thought-out business use case. I’m wondering how we can verify whether our hypothesis that we should build a piece of software using artificial intelligence is actually the right choice for a new project. How should we approach a proof of concept when we have a business problem and we’re trying to decide whether to solve it with AI or without it?

Bartłomiej Gładys: First of all, we always need to clearly define the objective of the project, because clients often come to us and say they want artificial intelligence, they want machine learning, because that’s what’s hot right now. But we need to identify an actual business objective. Then, as machine learning specialists, we have to translate that into what we call an objective function, which means defining that goal even more precisely. Next, we need to establish certain assumptions. And these assumptions are not just about what we receive from the client as input data, but also, for example, what exactly we agree on in terms of accuracy, costs, and performance. There are a lot of variables involved, and we need to lay them out with the client from the very beginning and be transparent about what it will look like and what the risks are. Those risks are particularly important because, in order to mitigate them, we usually create what we call a proof of concept. We try to quickly build a tool without an interface or any bells and whistles, simply to validate whether the idea can actually work. Then we decide how to move forward. There are also some useful tools that allow us to quickly add an interface later on. Streamlit, for example, is commonly used for this. Once we have a model, we can put together an interface in a day or two and show the client how it works. So it doesn’t have to be just us, as developers or machine learning specialists, looking at metrics, formulas, and so on. We can turn it into an interface and very quickly demonstrate it to the client.

Bartek Śliwa: Okay, so in very simple terms, the business process looks like this: someone who wants this kind of solution comes to Bartek Gładys and says, “Bartek, I have this particular problem.” And you might say, for example, that this isn’t actually an AI use case and that it should be solved with conventional programming. But if it is a use case for AI, you define what you called the objective function and then build a proof of concept.

Bartłomiej Gładys: A proof of concept, exactly. It’s also worth adding that in the age of language models, you often don’t even need to train the models yourself. Everyone talks about model training and says that every company needs to train its own models and so on, but often it’s enough to do something called database indexing. There are ready-made tools we can use in the context of artificial intelligence that allow us to index a database in a way that preserves the semantic meaning of its contents.

Bartek Śliwa: It would be great if you could translate that into business language right away, because not everyone will understand what database indexing means.

Bartłomiej Gładys: Basically, we turn our database into something like a search engine.

Bartek Śliwa: Okay.

Bartłomiej Gładys: And once we have a language model, which might be available through an API or a service like ChatGPT, that model can connect through our code, which is usually fairly small, to the search engine we’ve created. And that’s where you start to see the business potential. Something that used to take 15 minutes to find can suddenly take just a few seconds. This is also a fairly common use case right now. Many clients come to us because they want to search for information within their company more quickly.

Bartek Śliwa: Okay. Are there any key lessons you’ve learned from working with AI- and machine-learning-based systems, or any lessons that the recipients of your work have learned?

Bartłomiej Gładys: I think one of the key lessons is that you always need to be very specific about what the outcome of your machine learning work is going to be. At the beginning, it’s often some kind of analysis or a proof of concept rather than a finished product. That’s because the first thing we want to do is reduce the risk, see how things work, and make sure the client understands what that risk looks like at the initial stage. One way to define those objectives is by creating a test dataset. We can build a test dataset that both we and the client can see, and the client can then observe, iteration by iteration, how our model performs against it. That gives us a clear view of the direction we’re heading in. And we don’t have to be the only ones building that test dataset. The client can contribute as well because they understand their own business. They can show us what the correct result should look like for a particular case. For example, if we have a document-tagging system, the client can provide specific documents and assign the tags that should be applied to them. Then we test the model we’ve built, or the entire mechanism based on ChatGPT and a more intelligent search engine, against those test cases and see whether it performs correctly.

Another situation is a little more technical. A client will often come to us and say, “I have great data, it’s very well structured, it’s in a great format, so we should definitely be able to do something useful with it.” The problem is that sometimes what we actually want to determine is the cause of a particular outcome. Imagine, for example, that we want to determine the best prices for hotel rooms. The issue is that traditional machine learning often relies, simplifying things a little, on correlation. But correlation isn’t the same thing as cause and effect. There was a case described some time ago, I think somewhere in the U.S., where researchers tried to determine what was driving crime in different neighborhoods. They found that the areas with the most crime also had the most police officers. So they concluded that perhaps the police officers were causing the crime. Obviously, that wasn’t the case. The police officers were simply being sent to the areas where the crimes were happening. So these are the kinds of things we also have to consider when working with machine learning. You need to understand the nature of the problem. Are you trying to identify relationships within the data, or are you trying to determine whether there is a cause-and-effect relationship? Those are two completely different things. That’s also something we’ve run into in projects in the past. Now we understand that we need to go deeper into the problem and really understand what the client is trying to achieve.

Bartek Śliwa: Right. So the human element still can’t really be replaced here.

Bartłomiej Gładys: Exactly.

Bartek Śliwa: Because data is one thing, but you can still draw the wrong conclusions from it.

Bartłomiej Gładys: One more example just came to mind. Clients often have specialists in a particular field, right? And sometimes a client comes to us wanting data analysis, but if we don’t work with those specialists, we may get to the very end of the analysis, after putting in all that hard work, only to discover something they already knew very well. So the connection between the business side and developers is incredibly important. We need that collaboration to make sure we don’t end up doing work that turns out to be completely unnecessary.

Bartek Śliwa: Right, technology is supposed to support the business, not the other way around. I wanted to ask you about some of the buzzwords and myths surrounding AI, because obviously it’s a topic that gets a lot of attention. Quite often, when I talk to someone who knows much less about AI than you do and they say they’re using artificial intelligence, it turns out that’s not necessarily the case and they’re actually relying on certain buzzwords. So what’s true and what’s a myth here? What are some common misconceptions?

Bartłomiej Gładys: Going back to that last point, it used to be the case that someone had a few if statements somewhere in their code and called it AI. Or, as I mentioned earlier, there was some kind of imitation of intelligence because the system included an element of randomness. You could create a game, for example, where a character moves around randomly and call that artificial intelligence. That used to happen. But today these tools are so widely available that if someone says they’re using artificial intelligence, they probably really are. So that’s one thing that has changed.

When it comes to current buzzwords, I think a lot of people talk about hallucinations, for example. But these models work in such a way that, in a sense, they’re always hallucinating, and only sometimes do they manage to say something that is actually true. That’s because the whole process is based on continuously predicting the next word with a certain probability, one step at a time. That’s one thing. But when it comes to hallucinations, there are also ways of dealing with them. A lot of people describe hallucinations as a major blocker and say there’s nothing you can do about them. But there are ways to address the problem. For example, we can take a piece of text and, if we want the model to tell us something about it, ask the same question several times. If the answers differ significantly, then it’s likely that the kind of hallucination we’re talking about has occurred and the model is making up the information, because the answer changes each time. If the answers are the same, there’s a high probability that the model got that information from somewhere, because otherwise the data, and therefore the answer, would be different each time.

Another buzzword I think we hear a lot about is the number of tokens these models support. Everyone has probably heard Google say that a model now supports one million tokens, two million tokens, and that’s true.

Bartek Śliwa: What exactly are tokens? Let’s explain that.

Bartłomiej Gładys: Right, let me explain. In simple terms, we can think of a token as a piece of a word. Generally, before a language model is trained, a tokenizer is also trained in a certain way. It breaks the text down into tokens so that the model knows what it’s operating on. A token is then represented as a vector in a multidimensional mathematical space, and based on that representation, the model can understand what a given word or token represents and, more importantly, how it relates to other words. That’s a simplified explanation.

Bartek Śliwa: Okay.