A Beginner's Guide to Machine Learning with ml5.js

 A Beginner's Guide to Machine Learning with ml5.js - YouTube

https://www.youtube.com/watch?v=jmznx0Q1fP0

Transcript:

(00:00) - Hello! And welcome to a new playlist of videos. This is a playlist, hopefully there's a bunch of videos in this playlist because I'm just starting to record them now so they don't exist yet but this about machine learning and it is for the total beginner. Now there's some caveats here I suppose to that but I really wanted to make something, you don't have to have watched any other videos about machine learning on my channel, I have others that go more in depth and use different tools.

(00:25) This is really meant, if you're starting, it's totally new to you, you've heard the words machine learning, you've typed them into some learning machine, learning, aah, search bar and somehow you landed here watching from the future, leaning back, you've taken your brain out for its afternoon thawing and you're gelatinous couch is comfortably supporting your skeletal, okay anyway, the point is you're here, I'm here, hello.

(00:55) So I'm going to be using for this set of tutorials a pretty new JavaScript library called ml5. The purpose of this first video in the series is really just to talk about two things. What is machine learning? What is ml5? So the one caveat I would say, possibly, to being a beginner here is that you probably want to have some knowledge and experience with coding itself.

(01:20) Now that isn't to say that you aren't, if you don't, you're welcome here, please watch, I will point you to other videos that can help you fill in some missing gaps but I'm going to assume a little bit of knowledge about programming, particularly in JavaScript and I'm also going to be making fairly heavy use of another JavaScript library called p5.

(01:38) So if p5 is new to you, I'll refer you to some other p5 tutorials. I will but that's going to be a large piece of how I build various examples throughout all these videos. Okay so before I start talking to you about what is ml5, which is really what I'm here to do, let's just ask ourselves a question.

(01:57) First of all, it's called ml5, ml for machine learning, what is machine learning? Now there are lots of terminology out there. Machine learning, deep learning, artificial intelligence, neural networks, the list goes on and on and hopefully the definition that whatever I say to you right now doesn't really matter because we're going to sort of explore this topic of machine learning and all these other, all this other terminology throughout these videos bu I think one thing to realize about machine learning, you think of this idea

(02:24) of artificial intelligence, computers, robots taking over the world and they've learned their own language and they're defeating our, but you know, really, when it comes down to it, brass tacks so to speak, machine learning is examining some data, a whole lot of numbers and trying to make sense of that data and teaching a computer basically to learn patterns that we can present, that's really, and we're not talking about something called supervised learning here, whoa, even more terminology, supervised learning,

(02:57) I'm gonna get into all that and I've gotten into some of that in other videos. So I just pulled up some notes that I made for a course about artificial intelligence and at NYU here and this is sort of a classic definition of machine learning from 1959, Arthur Samuels, a field of study that gives computers the ability to learn without being explicitly programmed and so let's think about that for a second.

(03:25) This is actually useful and I'm gonna get into ml5 in a second but just humor me for a little bit longer. I might say something like the most simplistic sense, I could write a program that has a specific algorithm that says if mouseX >, so let's say this window has a width between zero and 400, its mouseX is > 200, then that is the right side.

(04:00) Can you see this? Yes. > 200 is the right side. Else, if mouseX is < 200 or just else that's the left side. This is me providing the computer, so to speak, I've written a program to some intelligence. It knows how to tell if a point is on the right side or the left side of the window. I have written the algorithm for that.

(04:24) Well machine learning is about teaching a system to understand a data without explicitly writing your own algorithm, this might be the most trivial, simplistic algorithm ever but the way I would do machine learning is I would say hey, machine learning system, I'm going to give you lots of information, like 350 is on the right side and 301 is on the right side and 212 is on the right side and 273 is on the right side and 100 is on the left side and 97 is on the left side and 24 is on the left side and 198 is on the left side. I'm creating a data set

(05:05) and I'm just gonna say here's a lot of stuff, I'm not gonna tell you how to, don't, I'm not gonna give you some specific hard coated algorithm to know what's on the right or the left, I'm just going to create a data set, it's going to have a lot of information in it and the machine is going to learn the patterns in that data, learn how to make some kind of prediction, that's a key word here, the prediction being is this point on the right or left based on that data.

(05:31) This is the essence of machine learning, teaching a computer to perform a task without explicitly programming the algorithm for that task. Now, it's a little convoluted 'cause aren't we programming the machine learning algorithm and that's sort of programming but this is the conceptual idea.

(05:49) Okay, so now that we, as best as I could, made some definition, created a definition of machine learning, one that at least we can start with and I'm sure I'll hear from you in the comments. Let's talk about what is ml5. So before I talk about what is ml5, I wanna just mention someone to you, Red Burns, who is the founder of the program where I currently teach called ITP.

(06:09) Red Burns is a huge source of inspiration, she was a mentor to me and I wanted to start say, I wanted to read a few quotes from her. I actually have a whole book of Red Burns quotes but these I think, these quotes are really important and they are the kind of driving principles behind a lot of the work that I do but in particular a lot of the reasons behind the ml5 project so, "technology is not enough.

(06:37) " "Consider the technology," I should really move the mouse. "Technology is not enough." "Consider the technology as a tool which, in itself, "could do nothing." "Treat the technology as something that everyone "on the team could learn, understand and explore freely.

(06:56) " So this is really important. Don't believe the hype. Is that an expression that people say? But it's easy to get caught up in the shining technology stuff but we have to remember that without human beings here on this planet, right, what's the point? So while I'm really interested and I get lost in my interesting just sort of numbers and algorithms and patterns and things like that, that's I think a sort of core value that I want to make sure I stay grounded to throughout anytime that I'm looking at algorithms and thinking about how

(07:30) to use technology, how to teach technology and how to make use of it. So this is actually from an article from the ITP Journal, Adjacent, there will be a link to this in this video's description which describes sort of the background about the machine, ml5 project itself.

(07:48) At the end of this video, I want to talk about various people who made contributions which I have here on this link so I will do that but let's talk about what is ml5. So in order to kind of figure out what ml5 is, I'm going to try to make some diagrams. This is gonna be great. I can feel it right now. So I'm gonna put ml5 right here in the center.

(08:11) Now, an important component of this discussion is a library called p5.js. Another really important component of this discussion, I'm gonna put it up here, is a library called TensorFlow.js and, oh boy, oh boy, another really important part of this discussion is just the library TensorFlow itself as well as you know maybe sort of this thing called Keris which I might refer to.

(08:46) So first of all, what is ml5.js? Ml5.js itself is just a JavaScript library. It is a library of functions that you can call from your own JavaScript code. So again, I mention that I'm gonna rely heavily on p5. P5 is also just a JavaScript library. P5 is a little bit of a special JavaScript library. Make a little heart there.

(09:16) A couple stars. Because it is really, it doesn't necessarily conform to the standard patterns you might find in other JavaScript libraries. It makes a lot of decisions to be as friendly and welcoming and well-suited for a beginner as it can be. So one of the things you can do in regular, in p5, is you can write a function called setup and any code that you put in setup executes the moment the program starts.

(09:45) You can write a function called draw and any code that you put in draw happens in a loop over and over again and this is a standard kind of design pattern for writing a program that generates an animation like a game or a piece of interactive art. So what I'm saying here is that my examples that I'm gonna show you in these video tutorials are using p5.

(10:10) js almost as like the base starting point and then I am also going to import the ml5 library so I need to import both of these libraries and I'll show you how to do that in the html file, which is something you'll see in the code examples and once I have the ml5 library, I can start to do things like this. Ml5 image classifier. So this is a function setup, is something that's sort of specific to p5, draw is something specific to p5 and then image classifier well we kind of know it's specific to ml5 because I had to write ml5.imageclassifier so this is one thing

(10:50) I'm a little bit off on a tangent here which is no surprise if you watch any of my videos before but a lot of JavaScript libraries are name spaced meaning if I'm gonna call a function like the image classification function, I've gotta say the name of the library dot that function name. So image classification, this is one of a common task, common machine learning task, right, I'm gonna say, "Hey, machine learning, here's some images "of kitty cats, here's some images of puppy dogs,

(11:19) "learn about them." And we're gonna see how you could do that exact kind of task with ml5 itself. Okay so this is the basic idea. (sighs) Now, why am I talking about this? So here's the thing. If you just were wandering around the streets of, I don't know, Venice, Italy, I don't know why I thought of that, they don't have streets, it's all, like, canals, oh my god, that's crazy.

(11:46) You're paddling through the canals of Venice, Italy and you go to the little canoe. Is that what they call it? No, the boat next to you, and you say, "Hey, do you know how to do machine learning?" Somebody will probably say, "Oh, you just go use "the thing called TensorFlow." Now it is possible they say something like, "Oh you just go use this other thing like pie torch," or I don't know, something else.

(12:08) But there are many, many examples of libraries, most of 'em open source libraries that have a lot of functionality for common machine learning tasks. One of the most well-known ones of those is called TensorFlow. This is a project from Google. It is open source and most of the time, if you are using it, you would be writing code in the Python programming language.

(12:37) Well in my tutorials, I will be writing code in JavaScript programming languages, programming language, not in Python. Well, amazingly, there happens to be this other thing called TensorFlow.js also developed by Google researchers, I will include a lot of information about how you can find out more information about TensorFlow.js and the creators of TensorFlow.

(13:00) js in the video's description but this is a version of TensorFlow written in JavaScript. Now in truth, TensorFlow really, by the way, is really running C++ code so even when you write your code in Python, it's just running other stuff that's C++ but that's not the case with TensorFlow.js. TensorFlow.

(13:18) js is everything's happening in JavaScript and ml5 and you could go and just use TensorFlow.js directly to do your machine learning stuff and I have some video tutorials that talk about that but ml5 is a project that actually wrap, that is a layer, it includes TensorFlow.js so ml5's a library where additional functions were added on top of TensorFlow.

(13:44) js to make certain tasks easier and in truth, this is a low level library. I mean if you were to use it directly, you have to really worry about things like memory management and this kind of stuff is eh, it's interesting, you could do it but if you're just getting started and want to learn the basics of machine learning and experiment and play, this is going to be in your way.

(14:08) It's gonna make things take longer and so ml5, the goal is to make a lot of the things that you're doing in TensorFlow.js a bit easier and more accessible. Okay so here's the other thing that I really want to discuss. There are three, I would say, I'm gonna give you, I'm gonna talk about three components of the field of machine learning right now.

(14:34) There are particular machine learning algorithms. One example of that is something that you might have heard of called a neural network. This is an algorithm to perform machine learning task. There are many other examples of algorithms. There are also, there's something known as machine learning models and in particular, there's this idea of a pre-trained model.

(15:00) So I want to talk about that in a second and then of course we have this idea of data and data sets. So I'm going to circle this here for a moment. I'm going to put my stars next to it because this is essentially where we will begin. You and me and the people from the future who are watching this video or that have time traveled in to the past, I don't know.

(15:29) Whatever. This is where we're gonna begin. Now, most machine, a lot of machine learning tutorials will start with something like and now we will look at linear regression, it's kind of somewhere in that thing where I was like, I had that line in the middle of the window or the points on the right or the left and we're gonna do linear, so there's a lot of mathematics and statistical components of machine learning algorithms but one of the things that is happening in the world today, the reason why I'm hoping that this video tutorial

(15:59) and this ml5 library is important and useful is that our lives are increasingly being governed by pre-trained models. Corporations train models to perform a particular task to recognize your face, to turn the words you're speaking into text, there are questionable, many questionably ethical applications of this.

(16:26) There are also fun, artistic and playful applications of this so one of the ways I think to get started as a beginner to machine learning is actually not to learn all the math and learn how to train all your own machine learning systems but to actually make use of some pre-trained models that are out there, for example, the thing that I'll start with in the next video is a model called mobile net.

(16:49) This is a model that's been trained to recognize the content of images but there are lots of images that I could show to that model that it will have no knowledge about and other ones that it will weirdly have a incredibly obscure detailed knowledge about. Why is that? Well what was the data that was used to train that model? What was the algorithm? What algorithm is that model using? These are the critical important questions that we of citizens of this world increasingly governed and controlled by software need to feel confident

(17:20) and capable of asking and so by taking, by friendly and accessible, approachable ways of being able to experiment with pre-trained models, this is a good place to start. All right so machine learning is a thing, it's the thing people are talking about. We wanna get some experience playing around with it, experimenting with it, understanding what it is.

(17:43) We're gonna use this ml5 library. I'm gonna show you how to import it. I'm gonna show you how to import it into a p5.js sketch and I'm gonna show you how to start to use pre-trained models and we're gonna look at what those models do well, what they don't do well, where do they come from and why do they exist, okay.

(17:59) So actually before I go, let's look at a couple more things. All right, so you know I've encourage you actually before you watch the next video, if you're continuing to watch these videos, to just check out the, go to ml5.js.org, I would encourage you to click on, there's lots of stuff to read about here but I encourage you to click on examples and go down and just try all of these.

(18:18) So you can see there's some pre-trained models that I'm gonna be using, I'm gonna do a tutorial basically about each one of these, notably I'll mention one called PoseNet which actually can take an image and estimate your pose, meaning where your hands, where your shoulders, where your eyes, where your hips, I might be getting that somewhat wrong there but that's a pre-trained model, just like a pre-trained model learns how to classify the content of an image, here's a pre-trained model that looks how

(18:46) to figure out where the human body is in a particular image. So we'll be looking at those, I encourage you to sort of click around here. I also want to just quickly go to this twitter thread that I posted, just to kind of give a little bit of background and particularly credit because ml5.js is not a library that I created, it's a project certainly that I have worked on but it came from many, many other people so first thing to mention, and I did talk about this, that ml5.

(19:15) js is a layer on top of TensorFlow.js so all of the heavy lifting behind the scenes really comes from this library which was, many people are working on and contributing to this library at Google and otherwise but the project was initiated by Nikhil and Daniel, researchers at Google. The development was funded by, with the support of the Google Big Picture Group, the Google Big Picture Group and the people at + AI Research Initiative which was led by Martin Wattenberg and Fernando Viegas.

(19:41) And they've done a lot of wonderful projects about visualization and machine learning. The style and approach of our library really comes from the processing foundation, I mentioned the relationship, I guess I meant to say the reason why the 5 is here is because of the sort of symbiotic relationship with p5, we've made sure, ml5 library, you absolutely do not need to use p5 with it but it is fully compatible with p5 and the spirit of p5 I think, in its welcoming and inclusive sensibility is a present in the ml5 library and so as a thank you to p5,

(20:28) we've used the 5 there. Okay. Cristobal Valenzuela who is also the creator of runwayml which you should definitely check out, started this project at ITP by creating a few examples with the deeplearn.js library which is what TensorFlow, which is what became TensorFlow.js. Yining has done amazing work on importing style transer, Yining she and pix2pix models into ml5.

(21:00) This is, tha's really wonderful. Pikachu, edge to edge Pikachu, Yining hopefully will either be making her own videos that I will point you to or I'll have her in as a guest to go over some of these. Hannah Davis has done a lot of work around data collection and responsible data collection that I encourage you to read about and find on the ml5 website through an artist residency.

(21:22) Nikita Huggins, a resident researcher at ITP really did a ton of work, project management work, vital, important work in shepherding the library through many different stages. The ml5 logo on the website is designed by Alejandro. Hayeon Hwang also gave lots of support to that. Dan Oved actually ported the PoseNet model to TensorFlow.js and ml5.

(21:51) js, I'll be showing this in some video tutorials in the future and the ITP Adjacent Journal which I did mention also, thank you to Stephanie Koltun who did all the editing for that article and I encourage you if you really want to read more about the background instead of listening to me ramble, you can check that out. We had a lot of guest artists who came to visit us and give us lots of inspiration and ideas throughout the process and then many organizations allowed us to do some of the first experimental workshops.

(22:17) School for Poetic Computation, Eyebeam, Creative Coding Fest, ITP camp and yeah. And there have been many other contributors that you can find if you go to ml5js.org and you click on code and browse the github repositories and see all the wonderful people who have contributed. So this has been my introduction to ml5, the library itself and guess what, if you made it through this video, congratulations because in all the rest of the videos, I'm really just going to be coding and talking through how to make certain examples, how they work

(22:54) and what the pieces of those examples are. So thanks for watching this first video and I'll see you in the next one which the first thing I'm gonna do is just make a simple image classification example using the mobile net model. See ya there, maybe. (upbeat music)