t-SNE Dimensionality Reduction with Scikit-Learn

NOVEMBER 21, 2022

How to encode an image dataset to reduce its dimensionality and visualize it in the 2D space.

Chapters

Transcript

Hi everyone.

It's Nono here and this is a hands-on tutorial on dimensionality reduction. We're going to try to do that with scikit-learn and the t-SNE implementation that comes in this package.

We'll drive to get the codings that we've obtained from another neural network, put our dataset inside of scikit-learn and try to reduce the dimensions of our dataset so we can plot it into a 2d map or an image, and we can understand the similarities or how our dataset looks when you compare one input to the other or one data sample to the other.

It's Nono Martinez Alonso here, so let's get started.

So I'm getting material from Hands-on Machine Learning with Scikit-learn Keras and TensorFlow book by Aurelien Geron that I'm encouraging you to get or, or like learn from if you want to get into machine learning specifically about Chapter 17 that was autoencoders and GANs.

There is a lot of theory there that, or even hands-on practice that we learned from.

There are many ways in which you can do this stacked autoencoders, but long story short, you try to input something and get the same in the other way, but on the way you're training an encoder. So you're obtaining these codings on the right, and then you have another second part of the network that is a decoder which from those codings tries to regenerate the original output.

These are networks that are mainly used for noise removal filters, not for generation because the output are usually noisy, but if you get GANs on the game and other types of networks, these are good architectures for generating images.

So is what we want, I would imagine that we can obtain this is a t-SNE flattening a highly dimensional space.

With these libraries, the t-SNE library from sklearn manifold. And this is really pixelated, because this is a really bad image, but I have the book here with me now.

So yeah, we're going to try to visualize that

Problem overview

Okay, so in live stream, so Live 33 and 34, we did see stacked autoencoders. We made uh, I believe uh, two versions of a Colab notebook.

In there, we were actually having that architecture in which we were taking in an image. So that would be like maybe a PNG I think there was 28 by 28. We were encoding that into 30 digits, right? So these were the codings of our network. And then we were getting that back into another image that is 25 by 25. That was the reconstruction.

And this was our input. So we try to do here is to train this part, which is an encoder. And uh, this part here that is a decoder. What the encoder learns um, how to do, right? So the encoder learns how to take an image. So whatever it is, and it's 28 by 28, which is 768 pixels into a 1-D tensor or array, that has just.. Is one by 30. So it has 30 digits, right? It's just 30 digits. It's not pixels because here actually we have RGB. So three channels for each of those pixels.

All right.. So we're trying to get, and I'm going to go to the next page now. I'm trying to get a network where we put an image, right? So we have our network here, which is the decoder and we're going to get, so this is our input and this is our codings. And those codings that I mentioned are going to be one 30. So 30 digits will then input into t-SNE and are going to be able to plot on a 2d image. And hopefully we're going to get clusters that belong to different classes.

So for example, this could be a type one or type two. The problem is that the data set that we're going to use those sketches that I showed before. Those don't really have a, like, we don't have a classifier to classify them. So we'll just see how the data samples look and the similarity between them.

So we're just going to try to do that. Let's just add here life 38. So this is like 38. 20 seconds of your life. All right. So I'm going to read here the comment from Ricardo. So I've already used these ne let me see. Oh, message. Redrafted. I was going to read it in regard to the, but you lead it before I could do that.

So if you want to rewrite it, if not, it's fine up to you. I was now I'm curious about where you were going to say that you've already used at t-SNE.

Okay. So let's keep going. We have a set of 105 images. We need to recover our notebook and uh, we need to make a notebook with sci kit learn.

Dataset

All right. So this is our dataset and these has 105 sketches from the website.

If you go into the live repository, so this is the repository of the live streams in livestream 35, you have a TypeScript script that will guide you step-by-step on how to use an, a synchronous pipeline to download those images from my RSS feed that is here, but you can actually change the URL of the RSS feed and a few properties to download images from another source.

In this case, we were just getting the images from here. So you can go there to take a look at that. I was mentioning, we have these 105 images. These ones are 2,500 pixels height.

So the purpose of having the images down to 28 pixels, is to be able to put them into the Keras model and have the training be a bit faster.

Steps

So format, data set, train stacked auto encoder um, dataset conversion to codings and then our codings, we're going to go into t-SNE and then plot them.

Colab notebook outline

All right. So you got Google Colab here and we can now access the stacked auto encoders notebook.

Let's see what parts of the code we need from the previous notebook.

Colab notebook outline

The first thing that we have is that we have the Fashion MNIST dataset, and I want to do this with our own data.

So what I'm going to do first is I'm going to add a text block here.

I'm going to say load to dataset. And this is like images are 20 by 20 pixels

RGB.. Then the second thing that I said is we need to define encoder and decoder.

Define stacked autoencoder then train model And then we'll have to do dimensionality redaction with t-SNE all right. Let's take a look at what we have. So we have TensorFlow 2.5 0 and later would have to encode the entire dataset. We have, Python 3.7, we have a CPU and we don't have any abuse. We can select the GPU here to make our notebook faster. So this is now going to allocate it for you and initialize a new runtime. So now we run this again. We're going to have the same version of TensorFlow from Python, so now we have a GPU device is a Tesla T4.

Uploading our dataset to Colab

Where do we have our dataset?

How are we going to put our dataset here?

And maybe the easiest is that we just drag and drop, because this is a really small data set. The problem is that these might, you know, require that we do that every single time that we want to do this. But I'm going to go into this sketches folder. I'm going to use this script to zip my file. So it's just, I go into the folder and I just call this, which is just zipping the file with the same name that the folder has. So maybe we just call it sketches. And this is 352 kbs. This is super small. We're going to simply drag and drop that into our notebook.

A reminder that files will get deleted when the runtime is recycled. So if you close your notebook, go away and then come back the data there is not going to be there. So there are ways in which you can download those files from other source or upload them somewhere.

I have a video here on how to autodownload files from Google Colab. And this is something that I really liked because that way you can run your notebook and then your files end up on your Downloads folder and you don't have to worry about that limitation.

Now we have this here and I guess we could unzip them.

Load custom data set. We're going to do some preparation here. We're in the content folder, which is this one here. So we can do unzip sketches.zip.

If we put here, the -q, these process is silent. So we don't see anything except for that.

One thing that we can do to avoid these sort of questions is we can just say, remove all the JPEGs in this folder and then unzip that way everything gets removed and then zipped again.

We can specify here to unzip on sketches that way we unzipped to a directory instead of to the main public. , Alright, so now we have our 28 by 28 pixel sketches here, which is our data set, which is super small.

Dataset loading

Now we need to see how to load that in the same data format that we had our dataset on the previous livestream.

So this notebook here, you can see, we're downloading with this function here. So tf. datasets.fashion_mnist.load_data. This gives us data in a really specific format. At 28, by 28 pixels, and there are 55,000. So we want to get our data in this same format, but in um, having less, less of them.

We only have 105 samples, which is going to overfit a lot, and this is not something that I had thought off, but at least we can set up this notebook so we load all of our data.

We X_train_full, y_tran_full, and there's the test.

Let's try to load our data with TensorFlow. Let's take a look here. So just to see if this works, so I'm going to import pathlib, and I'm going to call pathlib path sketches. And then here, we're going to do data there and then our pattern here is going to be just everything here with JPEG. All right. Let's see if that executes. All right. We get shapes, types, strings. This was good before, just that I don't think we even need these things if we just put here sketches, but keep it for later. Just in case I think I was doing something okay.

The problem is that these are strings and they don't have a shape. So we could say for image in print image.

So what we have here is a TensorSliceDataset, which doesn't have any shape because the type is string and with those train files. So we can say here, train files, and we can say here file, or we can print the file. And yeah, again, we get these binary string here that is just the path to the string.

So we need a function now to decode those images, right? So the code JPEG. We're going to define a function that is going to decode an input image, and convert the compressed stream to a 3d you in a tensor.

We're going to need to call on train files, we'll map, and then call a function that we haven't defined yet. That is load image drain and num_parallel_calls we're going to use AUTOTUNE .

And of course there's going to fail because that function doesn't exist. So we're loading the data set and we're going to define that load_image, train, function, load_image_train, , we get the file path. And, what we do is that input_image is going read the file. So the file path and that input image, will decode our input image.

And we're just going to return that input image. And.. We could do other processing here that I'll show here and we need now that decode_input_image function that I was going to define below before.

So we're going to get tf.image.decode_jpeg with that image data that we've thin. And we have three channels. What this is doing is that we're converting the compressed string to a 3d uint8 tensor.

And now what we do is we convert two floats in the zero to one range. So we use tf.image.convert_image_dtype, and we get a tf.float32.

Now we would maybe here resize the image, to the desired size, which you know, it's going to be here 28 by 28, but if we do this, then the data set doesn't need to infer the mid size and we can do other operations, so we can do here image width and image height. And antialias, we can choose here either true or false.

So hyper parameter here, I will put here our settings and we'll put here image with equals 28. Image height is going to also be 28. We're here, so we're loading and then we'll resize the image.

So there's going to be the train data, which we don't really have test data, so we might need to do some split later. But for now, let's see what we got here. So we got this train.. We have a Berlin, um, data set, with shapes 28 by 28 by 3.

There's one thing that is different here from the previous. So we were getting this shape, right? We were actually getting just black and whites. Well, I mean, We do one, the images to be black, to be color. But maybe that makes this problem a bit more complex, a bit different than what we had before and we'll have to change the neural network architecture. So we have 28, 28, 3. These are the shapes.. Train..

If we do a train_dataset and we run.. Train cache shuffle buffer size, which we need to decide how much.. Batch size.

All right. So This is going to ensure that we have more data samples and we now need to define these two hyper parameters. So let's say our batches are 32 and we're going to see now if this gives us something different. So repeat data set shapes. We can now do, for some sample in train_dataset.take(128). Let's do just 34 print(sample.shape).

All right. So we're getting batches. And so let's say we just get, 32 multiplied by 3 plus 8 that is 104, right? So all we can get here is these four batches. And then we reset because we only have 105 samples from the dataset. They shuffle and then they start again here.

Because 105 we cannot really well, 35. So our batch would be for example, 35 and now maybe we still get, oh, because of the buffer size 35

we still get 34.

All right, let's actually leave it here.

We can get many different batches, so we can save them batches and we'll have more more samples here. And um, we don't really need these data batched but we have an RGB image.

One thing that we can do here is so Keras to black and white.. RGB to gray scale.

So we can actually convert to grayscale. So tf.image.convert_to_grayscale(img).

All right, let's try that out. Now we do have the shape that we wanted. So now these are going to be the same proportions. So 28 by 28.

We can bring this all to all the other notebook. Let's just try to debug with this. All right. So we have this here. Okay. Import.. Maybe import numpy as np.

And here we have, for example, X_train_full.shape. Okay. So this is a 6,000 of 28 by 28. So we get the first one. What's the shape? 28 by 28. And we print it 28 by 28. So there's 28 items of 28 digits. If we do have something that is like our data, right? We have here. Hm. The sample 28, 28, 1. So these means let's, let's just print it. So Let's take is one batch. This is the opposite of expands.

We have this sample now here that we've reshaped. So we get 10 he just got all those. All right. We have a way in which we can do that. We probably one to, because this is returning the images and here we're decoding the input. So maybe input image here, we do the reshape. And we say that our input image is going to be reshaped to 28 by 28. Let's see if this works. All right. Nice. Now we don't need to do any reshaping, so we just print the sample shape. Our dataset has a set of images that are 28 by 28. And I guess individually, each of these is 32 by 28, 28, which we saw up there. What's the shape of this 28 by 28 and full is 6,000.

Alright, awesome. So we're loading our dataset in a good way that we can now use it.

Recap

Were trying to load a dataset to do a dimensionality reduction exercise with the t-SNE implementation that comes with scikit-learn.

So we're loading the images in 28 by 28 size. really small to see with your eyes, right?

Dataset loading (manually)

These are super, super um, small images that are pixelated and we drag and drop those to here as a zip file.

So we unzip, those files. This removes the folder and then unzips. Great.

So we load the data set here. We actually then need to do this here. We can put these functions.

We have the data set here.

Here we can load a set of things. So if I do train dataset, take 10 and I get the first one.

Let's comment this out. TakeDataset is not subscriptable. All right.

We made that on an object that is not going to stop anytime, so maybe just make it into awesome. So let's see shape, convert to tensor. All right. I don't know this. This seems a bit like a hack, but I might get there..

All I'm trying to do is get the, um, the, the data set that we have here. Right. So we have a data set and if we get the, let's say the, this is actually a batch on our data. So for image in batch, we can print the image and maybe we just stay one batch. So we have the image shape is there's 28 by 28.

And we could actually plot this. Let's see what we can get. I'm not sure if I can do here at prefetch.

Here I'm getting at least about, so we can say, can I get just the very first batch?

Okay.

All right. I think these might work for now.

Right.

TensorFlow dataset to NumPy array.. We have the train data set. We can train with that and it seems like that's going to be okay.

Dataset loading (without tf.data)

So let's say we just load the data manually.

So load images manually..

Import image. So we can do import glob, from glob import glob. We do glob sketches where get all the paths, right? So for file path in this, we can say our image is going to be image open file path, and we can print, for example, the image width.

Image size.

So it's 28 by 28.

Once we have the path, actually, even if we don't use PIL we do have this load image train, so we can do load image, train, and then we put here the file path, and then we can say print image shape.

So we get images are 28 by 28 as well. We could say, okay, are like, let's say X train full could be that. And we just do X_train_full append and this image. And then we print X train full shape, which I think we need to make into an np.array. Right. All right.

So we do get the shape that we wanted. We don't need to print here. So we're just appending. We don't need PIL. All right.

So this is what I wanted to do. I wanted to try to do it with with a tf data library.

And once we have.. These are also black and white. These are the type that we wanted. And so we could say data.. So dataset. So our data is going to be this right. And then we can say, okay, we have data on the shape of this. And because we don't have more data, we're going to have to say X train full is going to be so 105 .. Well a 104. I'm not sure why there's 104. We do here data and we get the first 104 multiplied by 80.

So let's do 84. So we leave 20 X_valid.

Alright. So X_train_full is going to be just... is the data and y_train_full.

So we get here, we get the X train full [0]. They're already converted to dtype. That is what they're doing here through flow 32 divided by 255. That's what we were doing here. So we don't need to do that. So all the data, and now we have X_train, y_train. So we get here X train..

So we're going to get here. Um, minus 20 date X_train_full.. X_train_full for minus 20. And these would be x valid. And we'll print here. X train X valid shapes. Shape.. Train.. Probably the other way around. All right. So we got the entire data set is this. Shape. Then we have all the data and then we have the training and the validation data set. Cool. So we can get the encoder decoder architecture.

So we're going to copy and paste here for expediency. Otherwise we're not going to get there. We have the encoder and we have the decoder and we have the stacked auto encoder, which we're going to paste here.

Awesome. So we have our stacked autoencoder, and now here, we define it... rounded accuracy, which is our custom metric. We compile the model with a binary cross entropy and this learning rate and our custom rounded accuracy metric.

So we could run everything above the cell, so run before.

We have our auto encoder. We defined our metric, compile this. We get a summary of the model and we have to train now. So this is the actual cell that trains and the input on the output data is the same, so the goal that we're trying to reach is the same. So we use the same input and output data set on training, right?

So this is super fast, so maybe we can train for more epochs. It seems we might be over fitting there because we don't really have too many samples, we just have 140 images.

We have gone through this and this, and now we need to see, we can get the codings.

So we could get a sub network of what we had before

So I believe we can pass an image through our stacked encoder. So we could. X trains zero.

All right. So if we pass the image through the stacked encoder. We're passing the first dimension or data set here, we're getting the codings, which are 30 digits. And that what, I'm not completely sure. So this is after training 40 epochs. Let's say we retrain, let's say we retrain for well, if I retrain now, I think continue from where we were.

So we have the same accuracy, so we might need to reset our model here, and train again. So you see that the accuracy starting from, or the run, starts from 53, 70 and 96. We get really fast. We get really high. So I'm just going to put here, one so we get just to this step and I'm going to go through all the cells again.

We have this model here and this was with 40 epochs. We're going to try to do it now, just with one epoch all right. So the encoder network, right, which we defined here. And we included here in the auto encoder has actually been trained and its weights have changed. So when we run this fit method, the autoencoder has inside the encoder. And we actually have this network here that can get this image, this input image that is 28 by 28 image and then generate the codings.

And what we have here now are 30 codings, right?

What we talked about before, so these are those 30 codings, right? We were talking about a 28 by 28 image that was passing through a network. So we were training in this loop of encoder decoder and in the middle, this encoder part ends up with 28 by 28 on the left and then 30 on the right. And we are actually seeing that here. So..

If we look at the shape of X train. So if we look here is 28 by 28, I put it through the encoder and as promised, if we get the shape of this.. Yeah. It's one by 30. So it's a linear tensor that has 30 digits inside of it and, you know, promise achieved.

If we change the the input, so I'm taking different images here, the codings on the bottom, slightly change. They change a bit because the input is a different image. They're different. So 1 73 and 1 52. So I'm going to go to 15 epochs and I'm going to rerun these cells and retrain, right? So we retrain there. Now our rounded accuracy is again at 98.16, which I believe is the same that we'll get if we go to 40 epochs and what I'm going to do is that I'm going to put here 15 epochs, but I speak to see something similar to below.

Let's do actually 10 epochs and redefine everything from scratch here.

So that's 10 epochs and we're going to put here at the net epochs, and now we're going to put, or the app works and I do think we need a larger dataset.

So 40 epochs around this minus one, everything.

One thing that we can do is that if we run this by the stacked decoder, right? So if I now get this output from here, so let's say the coding and I run this, and I print codings, you see them. So now I can pass the codings back here and we get back. So this is a generated image. You generate the image shape, right? So that's again a 28 by 28 image that is black and white. And if we get the first one of that batch, we get an image. So let's take a look at how to plot those because I have the code here.

Plotting 28x28 images and 30-digit codings

So plot image, we have a helper method.. Plot image, show reconstructions, right? We just need to put these helper and we'll have to do some cleanup, not going to do it in today, alright.

Helper here. So we can plot the image.

And I'm not sure if this changes anything. If I do here, predict.

I get a tensor. And then if I do predict I get something different, but it's at the end of the day, it's the same result. And then again an array as well. All right.

We wanted to plot this image, right?

So we're going to plot the generated image. Let's see what that does. Plot is not defined alright plt.. We have another dependency and then we go back and we got our plot image.

So its learning something, we got the um, the input that we're using here, we can also plot it.

This is the input that we're inputting to the network. And I think we also have something to plot the digits. Maybe not.. plot_digits... the codings.

So if we plot here those codings.. So let's say plot_digits and we do those codings. Zero.

I will probably have to put all of these dependencies in the same place. All right.

So those are my codings. So they go from... their 30 digits from zero to one and they're 30. And depending on when you load them. So like at what point or up there, what training they, they change.

All right.

Training for different steps

We defined the model. We do these definition here. We trained... summary... and then we're going to train for 10 epochs. So still the model always gets to the same place, right?

99.32 accuracy.

And that's 10 epochs so we still have the same input. We could maybe take this one or this one, right? These are different sketches. And we pass that through this one that says it was in 10 epochs and now we can plot our digits. It's slightly different than before we generate an image.

And now we predict, so this is a slightly different than before. And, we could actually just comment this out and we're going to take another image. So let's say you met image 80, so let's say input is going to be X train 40. So now we just do this. So you just get the input. We don't need to do a lot more input.

And now you can choose from here with a number. So we say you input a sketch 40 to we have that, and this doesn't do anything. Is any something. Now we plot our digits and now we reconstruct our image, which doesn't look anything like this at all. So I don't really understand what's going on with with the dataset.

It might be, It might be that it's overfitting and it is like learning something that it does something really similar for everything, for all the images.

The Fashion MNIST data set has 60,000. So that's probably one of the things why it is performing so much better, right?

So now we're going, after all this effort to load our costume dataset we're going to define here their data set. So they were going to have the same x train x valid dataset. I don't think we need this part. We do need this one. So we're going to load that.

We're going to load our model. We're going to define everything the same way. And we're going to train for 10 epochs. Let's see how long this takes. This takes a bit longer now because we have a bunch more images we have 55,000 images for training and 5,000 for validation in the, you should be thinking right now, the accuracy is just slowly going up.

It's not as fast as it was going up before. So we can see here, these number keeps changing and that's good, right? This means that there is some actual learning that is happening, not just the we are overfitting on a data set and learning a few things.

I would be interested in seeing this network... how many data points it needs to give any is like some information that makes sense.

All right, so the model is trained. Now we can boot input 14 here. That is this, that looks like a shoe. I also looked at the data set is inverted. So that also my make it learning a different way. So we get the codings, we still get a five out of the last thing, calls trigger, the function, retracing is expensive repeatedly.

All right. But we get our data in. Now we get our codings, these codings may a lot more sense. It's a bit more granular is not just like a lot of black. And we get our generated image here. All right. So it doesn't look anything like it, but at least it's a shoe.

So its learning something and it seems like we're able to reconstruct the output from what we have. Let's actually just train for 10 more epochs and and see what we get.

Just to let you know what that's training coding's here actually represent these shoe. If we modified these codings we're going to get a different image, that's like a puppeteer. Like we, we have an image, we're encoding it into this image and then we decode it back. So if we generate images like, and this one, we will get images like this.

There is a correlation, of course, because of the weights that the neural network has learned from the codings to the output. It seems like we're almost done. We have a bit more accuracy, not too much more, and we are going to use the same input. I'm going to do one thing. I'm going to leave this here and I'm going to compare so how the codings are different.

They are almost the same. So say let's regenerate. This is in case generate the codings. They seem to be almost the same. Let's see if the decoder has learned anything else. So we have the image here. We're going to predict. And now, so I'm going to put it here. So I'm going to regenerate a new image and I can already see some changes in there

I'm gonna say, I made an error. I made a mistake, this has changed, right? So we can compare these two images and they're different. So there is something else that the network is learning while we're retraining. What is funny is that I forgot to change this variable here. So these should actually say input because we were continuously using extreme zero while we were visualizing here, the number 14. So if I actually do this right now, you're going to see how

the codings that we get are going to be super different. So look at the codings though. So these codings here are super different, right? These now doing code this shoe, we can see is they will go to zero and we plot that image. We indeed have something super similar to that booth that we were generating before. Which is, it's super funny. So we're going to go back to the shoe. I'm going to go back and generate the codings in.

Now I'm going to do any of that shoe and I need to predict before, so predict and in that shoe. Awesome. So now our decoder knows how to generate a shoe. So what these brings us or where these brings us it's actually trained for 50 more epochs, something that is going to take a bit.

And let's try to take the around the accuracy up. Let's see how the network learns differently. And later we're going to go back or we're going to use the data set of the sketches and actually verify if there is something that is being learned. Because before, even though we were training, we were not changing the actual input that we were predicting with on the encoder.

The codings were not really changing. So I do still have hope that we are going to see some some things there and that this is going to improve of it. All right. Maybe I can, yeah, no, I got to do that. All right?

Coding and decoding

Okay, we've trained for, I believe 50 more epochs, right? I think, I don't know, might be 10 plus 15 so we can see those here. So the codings do change a bit.

And now when we plot them, so these were the ones from before. So when we plot them, they're really different. And let's see what the generated image looks like now. So we put a block in here. We need to define this here. 14. We put this here. We plot our codings.

We do this huh? What am I doing wrong now? Input. Oh, okay. So same error as before. So we left these on the old code, so that's why we were erroring here. So we get the codings, the, these look a bit more like the ones below these look back again differently. And now we predict and now we plot.

Okay. So we're getting, you know, you see here a stripe and I would say it's hard to notice which one looks more similar, this is a bit more refined because we've trained for longer. So I would say maybe this one, so this is like, uh, 10 this is.. Uh, 10 plus 50. Let me show you here. And this is the input is the original.

So this is the actual input. So which one looks more similar? Not sure.

They're more or less the same.

So what I was mentioning before. So if we try, what I said is that these codings, right? If we look at the codings, what they really are, they're not more than an array containing side of an array, so we could get this here and we could say, my codings could be an np array with this data, and now we're putting the codings and now we get exactly the same, but this is float 32 type. And we could probably do that, as type.

So I get the codings these way. I plot them. They're the same because I just copied. And then we predict we get the image, right? Let's put that image below. So these are the same image and I'm going to bring these.. We'll leave them here.

So now what I'm going to do is I have these codings here and I'm going to copy them. So I'm going to change them them a lot. So maybe we put here a four, this is going to be an eight, is going to be minus four is going to be 13 is going to be one, right?

So we changed those codings. And now instead of the one we have above, we have the one below. And when I generate the image, that is what happens. We've changed the things. And there are some parts of the image that still look the same, but by changing numbers, we're changing different features on this image.

And one can imagine that now, if I do put this one here to zero and I plot again. Now we have just changed slightly something here, right? So instead of Y we get these gray instead of these gray, we get this white and now I can predict again and generate the image. And now the change is back again and loose.

Again, more like a shoe, just changing one number, like changes the app a lot . So what we're going to make sure that we don't get confused later. And we just use the actual prediction function, and we're going to go back to uncomment to use fashion. MNIST data set, instead right?

So we comment this out and now what we just want to go back to where we are. I think everything right. So we don't really need to expand twice here and we don't really need two codings and we don't really need three visualizations.

Training on the sketches dataset

So what we're going to do is are we going to go back here? We're going to run, restart and run all. So these are going to restart the runtime of our Jupyter Notebook in Colab and it's going to execute all of our cells once again. And yeah, hopefully we can see something with our dataset.

So these just trained. We're at the same accuracy that we were able to get before. But now we have an image that if I said here 60 we can see a different input. So maybe we can just merge these ones because it doesn't, we don't really care that these are just one cell. So we're going to remove these other two. So let's see six. Okay. So this is different, right? From everything else. So we're gonna go and we're going to run the codings, which are different.

This doesn't seem like it's changing a lot. So we go now with 16, we have the input. We're actually predicting on the input. We plot the digits here, so we're going to plot the digits again and they barely change. So we do this and we do this. Yeah. So this is changing, but it's barely changing it.

Just learning like a real weird mapping. Let's do here. Save this as X train sketches. X valid sketches equals X train X valid.

All right. So we're going to rerun everything. So we're going to retrain now and regenerate the images.

So it seems fairly simple. So we have t-SNE the scikit-learn implementation, which is including a scaler, manifold, t-SNE. And all we need to do is something like this, more or less we're going to do that coding before we go today. And let's see how this went. . So we have these pants and we get pants, which is pretty good.

We have a lot of data involved in, in this experiment. Right. We could get these again, like hand generated, codings, which match, and this one's here pretty similar and we get this, which is a completely different output. Again, let's comment that up. And instead of getting an input X train, so we're going to get an input X train sketches, and we're going to get the fifth sketch.

So now we're talking about this one and the one thing that I mentioned before is that the inversion of those. So let's actually put this separate. So we put other logic here current dataset or force custom loaded data set.

All right. So we are going to load that and we're going to plot, so they input minus one. These would load the original image and this one would load like a separate sketch. The interesting part here is that we have images that correspond to my drawings and not to clothing. We want to see if we obtain the codings of one of my sketches, what is the the corresponding image from the decoder that we're going to obtain?

All right. This one, for example. So I think what we need to do is say here, input equals one minus input to get it negative. So we don't have to this here, and now we can do this. We can extract the codings once again, and we generate the image and then we plot it. All right. So you can see, right? Like it's activating that part here, so it's starting to make it into a shirt and um, it's a good experiment is kind of activating or encoding with one data set and then obtaining with another one.

Good thing is, if we just come in this out, we still have the proper data that we needed.

Dataset scatterplot with scikit-learn t-SNE

We just need to say, Okay, from sklearn_manifold, what we saw before import t-SNE is a, we import that library and now we say X_valid because otherwise we're going to have too many.

We say stacked and coder predict X,valid. And we do t-SNE equals t-SNE, so we instantiate this library and then we do X_valid to the equals, t-SNE fit transform X_valid compressed. All right. So I'm going to run this and this is, this is processing. And the next thing that we're going to do is that we'll plot make a scatterplot with the X valid to the, everything in the zero X valid to the, everything on the one C Y valid.

Do we have Y valid? I don't know if I remove that before. Okay. So we need that for some reason. And we just need here as then cmap tab10. Yeah. And these are the categories because we know the categories and we know that they're 10 different classes, so scholar shape, we have list of colors or color options.

Scholar, a scholar or re like shape option the marker size in points. All right, cool. Let's try this X._valid is not defined. All right.

Cool. All right. So these actually matches in some ways kind of changing the axis, what I have on the um, on the book. So I think maybe if I changed.. Yeah it's changing something on the axis. I don't know why, but it matches what I have the book. Yeah, so we didn't have, we didn't really have the core, so the valid.

So if we print here just so you see why_valid and I take like maybe 10, right? These are the classes of the data, right? So we have here X _valid. And if I take 10 you actually have the images, right? These are the images that were supposedly plotted in there.

And then here we have the colors. If I were not to provide that, we'll just get a set of, of um, blobs right. And you have to think that those have similarities among them. So this is t-SNE, right? This is the basis of what we were looking at.

I want to continue farther with this... so continue. I'm looking at this a bit more and developing be this part of that, because right now we just see like plots here. We don't even see images or anything, and it would be good to plot some images here, like even have an interactive board where we display the the samples and.. What else can we do? Can I try once more? We're going to train for... this is super fast. So let's do a hundred epochs and we're not going to have X valid this time. So we're just going to remove these part and we're going to reset and run all.

We're going to be looking at more dimensionality reduction algorithms. All right. So this goes here. Okay. What does this mean? X,_valid, I don't really want X_valid here. We want the actual thing, so let's actually do here

X train, compressed X train to the X train and that should be it. All right. So it seems like it's still it's fitting some in some way, these these drawings by similarity, I don't know that they're that different for them to learn any other things, they're spread on space, let's say. And and I really want to now see the plots in these individual sketches and see which ones they are.

Can I provide here a... like an array or something? One like 0 1, 2.

I don't remember now what the shape of this for us? 84 with X, some Y it says 84. All right. So range. Let's actually make a list, manually. So colors equal that and for i in range 84 colors append i colors. Let's see. All right. So we get 83 and then we can put that here. Colors. We get 84 colors, and that plots. All right.

That is useless. But we're going to do something just at least to cluster them. So we're going to say i = 0 and i plus one. And then when i, if i is greater than 9 i is going to be zero.

And j is going to go up sped j + 1. So we have less colors.

So let's see how those colors play out. Great. We have too many colors still, so we might just need, let's say we just have three clusters. So 84 divided by three as 28. So is it 25, right? We have now three colors. So it means that this is telling us that drawings that are not initially together, right? That this is not the order. They're like being clustered in different ways.

What do we have here?

Render images in the scatterplot

Visualize scatter with images. I start to just copy paste this one, which you shouldn't really copy and paste code, but because we're finishing up, let's see if we get that stretch goal. All right. So we got this here and we have this, so annotation bounding box um, offset image.

Okay. Let's see. So we can define this.

So visualize scatter with images. Let's see.

Oh, that is awesome. All right. So copy pasting. We got a pretty good resolution image, I think. And I might be able to maybe bump this up. Can I reshape array into shape?

Oh, reshape not rescale. Okay. You meant soon. Let's do maybe four because this is... these are too small. Great. This is pretty awesome.

So maybe let's just do two. I don't know why it is coloring like that..

Oh this is looking super good!

All right. We got it.

So, um, I don't, I don't think we're gonna, you know, we're going to get too far with this, but.. Long story short, I think we got it. We got what we wanted. So it seems in some way it doesn't seem like it lend to many different features. It does seem like the darkest ones are, or like the more infield ones on the bottom left.

These ones are the ones that are from some like people here. I think it would be a bit crazy if we do that with a segment of, of the other ones. So let's try to just copy these t-SNE sketches and we're now going to copy these and paste them. I don't know if I can paste them here.

Render t-SNE images of Fashion MNIST. Okay. All right. So how are we going to do this? We can remove this. We don't really need to bring this anymore. We do have this.. Put this here.. This here... and we do want the colors. We don't really need this anymore.

And X train 2d, and X_train maybe we're going to have more images, X,_valid maybe let's just pick 500 images for now.

We need to train the model and that's going to take a bit of time. So let's put this back. Um, We're starting to run out. There'll be things that are not going to work, but we'll get there. And this probably the thing that's gonna take the longest so we'll start again.

All right. So we're running again. I think we're still using GPU and you should get Colab Pro. If you're going to have, you're going to be training models for long call up prompt is 10 bucks a month, so definitely worth it. All right. So we're training five... six out of 10.

We're almost there. So we should be now done with training. We are.. Great. And we are probably going to see some error below, but let's see. Probably not. I don't think this is going to get completed. This is too much to be plotted.

All right. So I'm going to interrupt that one because where I want to really get is here. So we do that and then visualize. Okay. And we run this helper and then we run the cell once again, X train 2d X valid 2d. Alright, whatever.

So we do here cmap=gray. Run that again. And we're going to do that here. cmap=gray_r

This is indeed. Um, pretty, Pretty cool.

Okay. What this tells me is that, you know, we haven't trained for long enough, probably I'm going to get more samples. So I'm going to take 2,500. So let's see. Maybe it's a bit too much. Okay. That works. That's a cool image, right?

I don't think these ones are similar. I'm not sure if we're doing anything wrong here. Let me see if there's anything. So X valid X valid compressed X. It X train alright. X_valid. Images. Okay. So we're X_valid to 500, right? Let's see.

Yeah. This is like looks wrong definitely.

Okay. That's so much better. That's awesome. Okay. So this is, this is really cool. So this is awesome that just with ten epochs this looks so good. . So let's parametrize this then, because this, maybe we can do even better.

So we'll do input. It's going to be this and we'll just predict on the input and maybe its inputs. All right. I'm going to generate this just to make sure that we're getting the same. So zero zero, zero one, and uh, what we're gonna do right now is I'm going to close this thing.

So comment. This is being plotted, which is pretty cool. It has not changed. I'm not sure why it changed for some weird reason. And we're just going to train for longer just to see how that image actually changes how the sorting changes. So we're going to train here for 20 epochs. And when we have the model trained, these will be trained for 30 epochs total, and then we'll be able to get this thing.

Let's check that the images on the other data set are also..

Yeah. So I think this was okay. So we have the X train, an X train.

And now this should be running, so this is the comparison moment. So we're going to take a look at whether training for 20 more eepochs is improving anything of our result. And maybe we have the axis changed again. I don't understand why.

All right. So t-SNE doesn't give us the same axis all the time. It sucks. Hmm. So I can not compare,

So we're sorting shoes by similarity. So now, you know, we could create a recommendation system just with these images where if you pick this dress, you could propose a few other things that are around it. Great.

Call to Action

Support the channel by liking the video, if you're enjoying, or if you just want to support the work I'm doing here and subscribe, if you want to get notified, when I go live next or when I upload new videos.

It was Nono Martinez Alonso here. Thanks a lot for being there. I would have, I appreciate your comments.

If you have any questions, any suggestions, leave them on the video comments and um, yeah, I'll see you next time. Bye.

Blog Video Code Python Ml Tensorflow Scikit-Learn