Insights In AI and Data Science (Episode 5): Q and A with Aurélien Géron, Former YouTube Product Manager
🕔Read Time: 12 Minutes
Hello, community!
We understand these are trying times the world-over but we hope you are staying safe. Remember to follow recommendations from recognized health authorities in your region and help flatten the curve.
We are back to our "Q 'n' A With Our Experts" blog-post series. You can check the previous blog post in this series on "Your First Machine Learning Project" with Sayak Paul (Google Developer Expert) by clicking here.
These "Q 'n' A" series will help you get to know various practitioners and experts in the field of Artificial Intelligence and Machine Learning at a personal and career level; these are mostly non-technical, insightful and fun conversations.
Check the table of contents below to quickly jump to a section in the article.
Table of Contents
Learn from Aurélien's workshop on Natual Language Processing with the Port Harcourt School of AI. You should follow our Twitter page and subscribe to our YouTube channel to watch the premiere on the 24th of April by 1 PM (GMT +1).
Who is The Interviewee For This Episode?
In this episode, our interviewee is one of the guest facilitators for the Practical Machine Learning course from the Port Harcourt School of AI.
This interview will give you an insight into the mind of Aurélien Géron. Aurélien led the team that worked on the classification algorithm for YouTube while working as the product manager. He is perhaps popularly known for his best-selling Machine Learning book "Hands-On Machine Learning with Scikit-Learn and TensorFlow" which recently got a newer edition upgrade here that includes the Keras library.
Aurélien shares insights on his life and childhood, what makes a Machine Learning Engineer and Data Scientist world-class, and how he overcame challenges he has faced as an ML Engineer. You will find a lot of useful and unconventional tips in this interview.
What You Will Learn From This Interview?
Did you know Aurélien once lived in the city of Port Harcourt, Nigeria? 😱. You will learn;
Aurélien's golden advice for beginners if they are just starting out in Machine Learning.
How Aurélien's biology diploma was relevant to him becoming a Machine Learning Engineer.
How Aurélien learns new concepts in AI and Machine Learning and how you can too.
How your willingness to keep learning and your ability to follow a good methodology can make you a better Machine Learning Engineer.
How Aurélien became interested in the field of Machine Learning and AI and how he overcame challenges he faced as a beginner.
Main Points We Got From The Article
Here are the main points we got from this episode with Aurélien;
- As a Data Scientist or Machine Learning Engineer, Aurélien states that your most important job is to understand how to frame problems you are trying to solve so that they optimize for the most beneficial metric. Be clear about what you are trying to build and how it can directly achieve the business objective (or end objective). He explains more in the interview section.
- According to Aurélien, the top qualities to have to become a world-class Machine Learning Engineer are your willingness to keep learning and your ability to follow good methodologies. The interview gives context to this point.
- According to Aurélien, joining an AI and data community is one of the best career decisions you will ever make. We surely approve of that! 😁
Interview Section With Aurélien Geron
Personal Questions (Get to Know Aurélien)
Stephen: Hi, Aurélien! Thanks for doing this Q ’n’ A session with us — we are more than honoured to have you come on-board with this and contribute to our community. Please, can you introduce yourself?
Aurélien: My name is Aurélien Geron, and I am currently a freelance Machine Learning consultant, based in Auckland, New Zealand.
Stephen: Awesome! I am sorry I have to be a fan-boy right now, but you are one of my best authors ever 😁 and I am super glad to be having this discussion with you.
I am sure the readers are curious, what did you want to be when you went to college? What did you end up studying? And how did what you studied impact how you currently work in your present field?
Aurélien: Wow, thanks for your kind words, Stephen! :) Ok, let's start at the beginning. I was born in Paris, France. My father worked as an engineer for Liquid Air, an industrial gas company, and his role led us to travel quite a lot. We lived for about 3 years in Port-Harcourt when I was a little boy. Then 3 years in France, 3 years in Auckland, New Zealand, 2 years in Berkeley, California, and then quite a few years in Paris.
Since I was 9, I enjoyed programming, first in Basic, then Pascal. Later in my studies, I enjoyed Smalltalk, LISP, Prolog, then professionally C/C++, Java, and Python, with a bit of Javascript and Ruby along the way.
I studied micro-biology, and I hesitated to become a researcher in microbial evolution (I am a big fan of Darwinian evolution). But I discovered neural networks at that time, and I fell in love with them, so I decided to pursue a career in Computer Science. I got a masters degree in Computer Science, but neural networks had fallen out of fashion, so I started working in an IT consulting firm in Paris, as a software engineer consultant and a trainer. My biggest contracts were with banks, and between contracts, I taught C/C++, Java, and distributed programming technologies. I also wrote a couple of books (one on C/C++ and one on Internet/Intranet architectures).
Then my wife and I moved to Montreal, Canada, for 3 years. I was a software engineer in a telecom startup. It was fun and interesting, but it went bankrupt when the telecom bubble burst in 2001. I then became a freelance consultant, and my main contract was with the department of defense of Canada.
Then we came back to France, and I founded a consulting firm and a telecom company with my brother and a friend. I was the CTO in both these companies. I also wrote a book about WiFi deployment and security. Both companies are now pretty successful companies, with hundreds of employees and tens of millions of dollars of annual revenue. But after 11 years, I got a bit bored so I left them in 2013.
I joined Google in 2013, to lead YouTube's video classification team in Paris. I did that for 3 years, and it was great, but I was not getting my hands dirty enough (not enough code, too many meetings), so in 2016 I moved on to be a Machine Learning consultant. I wrote the O'Reilly book Hands-On Machine Learning with Scikit-Learn and TensorFlow, which has worked out far beyond my expectations.
Last year, I moved to Singapore with my wife and our 3 children. I worked as an ML consultant and I wrote the 2nd edition of my book. In January I moved to New Zealand.
To answer your last question: I studied both micro-biology and Computer Science, it's a fairly unusual combination. I think studying biology trained me to deal with complex systems (and even chaotic systems). In biology, you rarely expect anything to work out according to a strict and precise rule. There's always room for significant error. So you get used to accepting solutions that work "well enough", and you don't really strive for perfection. Similarly, I'm quite happy to train and deploy neural networks, as long as they work pretty well, even if the underlying mathematical assumptions are not always perfectly justified. This might not sound very positive, but I think it's a good mindset to have when you want to create startups, where it's more important to find quick solutions rather than perfect solutions. More generally, I think it's the right way to deal with complex real-life problems.
Neural network and the human brain illustration..
Biology is also full of emergent properties (i.e., properties of systems which cannot be trivially inferred from the properties of their parts, such as thousands of ants, each using simple rules, and yet working together to create a complex anthill). Our thoughts are emergent properties of our brains. This means that reductionism might not always be the best approach to study neural networks. For example, one might think that large neural nets can be decomposed into smaller nets, and each sub-net can be studied separately. But if the behaviour of the larger net is emergent, then cutting it into pieces is likely to break the emergent property. For example, small networks are prone to lead to local optima during Gradient Descent, while larger networks are not (or much less).
Lastly, knowing that the structure of the human cortex (the outer layer of our brain involved in conscious thought, perception, and more) is fairly homogeneous and repetitive, and knowing that there's very little difference between human DNA and chimpanzee DNA, I believe that our human intelligence is the complex result of simple rules (just like a tree looks complex, but when you zoom in, it's just the same branching pattern repeated over and over).
I wouldn't be surprised that Artificial General Intelligence will one day be implemented with a program of less than a dozen pages of code (for example, consider how incredibly short and simple DeepMind's AlphaZero is, and yet it beat the world champion at the game of Go, which many regarded as one of the hardest games in the world).
To this day, I continue to love biology, especially Darwinian evolution and micro-biology. I highly recommend reading The Selfish Gene and The Extended Phenotype, by Richard Dawkins, and all the natural history books by Stephen Jay Gould (such as The Panda's Thumb or The Flamingo's Smile).
Stephen: Oh wow, Aurélien. This is a lot to unpack. First, I could not contain my excitement when you stated you lived in Port Harcourt for a short while; this is so amazing! A lot has definitely changed since the last time you were in Port Harcourt but as a city, I am sure we are glad you once lived here. Thank you for the very detailed and resourceful response to the last question.
We will now move on to the next section, but you can pause and reflect on what you have learned from Aurélien's life in this section;
Pause and Reflect
Based on Aurélien's perspective, do you think you have to be extremely good at Maths to get your Machine Learning algorithms (neural networks) to work?
Can you see the close relationship between the complex systems in Biology and how that relates to Machine Learning?
Let us know your thoughts in the comments at the end of the article.
Applied Machine Learning and Career-Related Questions
Stephen: Welcome to this section of questions where we explore Aurélien's thoughts on applied (Machine Learning) ML and his career journey. Aurélien, what prompted you to dive deeper into the Machine Learning and Deep Learning field?
Aurélien: I loved Sci-Fi as a child, especially Isaac Asimov. I particularly loved his "Robots" series. I dreamt of one day chatting with a truly intelligent robot, and in my wildest dreams, I would help build it. In Asimov's novels, the 3 rules of robotics would protect humans from the robots, so it would not end like in Terminator.
Later, while studying the behaviour of insects, we used simple dense networks to model their behaviour. I fell in love with neural nets right there. However, I started working in 1997, and neural nets were definitely not in fashion back then, so I only played around with them for fun, not professionally. This changed when I joined Google.
Photo by Nice M Nshuti on Unsplash.
Stephen: The love of Sci-Fi and the dream to build Artificial General Intelligence (AGI) surely introduced a lot of other practitioners like you to the field. It's been a great journey so far I must say, Aurélien.
Pretend I’m not a tech person or Data Scientist. Can you explain Data Science/Machine Learning in simple terms and how it aids problem-solving?
Aurélien: Some problems are so complex that we don't know how to break them down into a set of simple rules that we can program in a classical way (such as image classification or voice recognition). Or perhaps we know, but it's very long, inconvenient, and hard to maintain (for example, we could probably create a fairly good spam classifier using only hard-coded rules, but it would be horribly long and difficult). Or perhaps the rules keep changing all the time (for example, to fight against spam using a classical program full of hard-coded rules, you would have to update the rules all the time, whenever a new type of spam arises).
Machine Learning allows us to write programs that learn the rules on their own, so we can tackle complex problems with a fairly simple program, easier to write and maintain, and easier to adapt to new situations.
Data Science is about extracting useful insights and knowledge from data. For example, a company might use Data Science tools to better understand its clients, and perhaps break them down into different segments (such as casual clients and passionate clients). This would help the company improve its strategy.
Machine Learning is good at detecting patterns in data, which makes it useful to extract insights and knowledge from data. That's why ML is often considered part of Data Science.
Photo by Luke Chesser on Unsplash.
Stephen: Short and profound distinction, Aurélien. Thank you for the clarification. I think this is an important question to address; what is one thing about Data Science and Machine Learning that is clear to you but is not so to other people (engineers and non-engineers alike)?
Aurélien: That's a difficult question. I think the worst error I saw software engineers make in the ML projects I worked on (and at least once I was guilty of this myself) was to lose focus of the final goal (or never really define it clearly), and to waste time improving the system based on the wrong metric. You don't want to spend several months working on a project that is unlikely to significantly improve your key business metrics.
Stephen: Awesome, that's definitely one of our take-home points from this article. Most of our readers are either beginners or practitioners and they face various types of problems and challenges when they either started out in the field of ML or when they start out projects. Did you face any of these challenges? If yes, how did you overcome them?
Aurélien: Labelling was definitely one of the trickiest parts of my ML projects. The trickiest, in fact. In some cases, you can label the dataset using a previous system. In others, you have to label everything using domain experts. In others, you have to rely on crowdsourcing services like Amazon Mechanical Turk.
If I focus more specifically on the challenges I faced when I started, I would point back to my previous answer: make sure you're always very clear about what you are trying to achieve, and make sure that the projects you prioritize are the ones that will most likely lead to your business objectives.
Stephen: Absolutely! Labelling datasets is definitely a huge ML problem while starting out projects. Getting labels is one of the crucial aspects of formulating ML problems (the other two are; choosing an objective and choosing input features). May I also add; in addition to using labelling services like Amazon Mechanical Turk, you can also use proxy labels (labels that measure something related to the true label) or build a labelling system to collect data in order to prepare for ML tech.
It's time to pause and reflect on what you have learned so far...
Pause and Reflect
Based on Aurélien's explanation, why do you think Machine Learning is also classed as Data Science?
How can you help yourself select the right metrics when working on Machine Learning projects?
Do you agree your method of getting labels depends on the business context and problem?
Let us know your thoughts in the comments at the end of the article.
Stephen: Welcome back! Let's get back to learning from Aurélien. What qualities do you think are most important for someone to become a world-class Machine Learning engineer?
Aurélien: Learning all the time, getting your hands dirty, being patient and most importantly, having a good methodology. There are so many steps in ML projects, you need to make sure you follow a good methodology, and you don't just go in random directions.
Stephen: Oh it had to be "learning", again! If you have been following our series of interviews, you will realize that the common reply to this question is "learning" which tells you a lot about what truly makes a Machine Learning Engineer world-class. Thanks, Aurélien.
Community members learning from one of Port Harcourt School of AI meetups from 2018.
Stephen: Aurélien, learning a new concept in ML and AI can be pretty difficult, Would you like to share how do you approach that process?
Aurélien: I often find a good blog and/or a good video to understand the main ideas. If this is not sufficient, I go ahead and read the paper (it's often much easier than it sounds, especially if you already understood the main ideas). Then I try to actually implement it, whenever I can.
And lastly, and perhaps most importantly, I try to explain it to someone else (or upload a Jupyter notebook or a YouTube video, but you could also write a blog post). This last step is important for three reasons: first, sharing what you learned is helpful for others, and it gives back to the community; second, it gives you a sense of purpose and belonging, which is very motivating; third, it pushes you to understand the topic in much more depth.
Stephen: Absolutely! I also support the claim that trying to explain a new concept you are learning is one of the best ways to deepen your knowledge about that topic. It's one of those famous Richard Feynman's learning techniques, which is very effective.
As we all know the AI field is rapidly advancing, Aurélien. What are you excited about in the future of Machine Learning and AI?
Aurélien: There are too many great developments to cite them all, but I am particularly interested in François Chollet's "On the measure of Intelligence" paper, and the accompanying Abstract Reasoning Corpus (ARC) dataset.
I think he (François Chollet) is really onto something: if we are to solve AGI, we need to better understand how humans manage to infer the right rule when given only 2 or 3 examples. I believe this requires some level of symbolism. You can think of this as discrete representations (i.e., digital), not just continuous representations like embeddings (i.e., analog). It also requires a better understanding of human priors.
The notion of a solid object and its physical properties (such as the fact that it cannot teleport) are probably hard-coded in our brains (by evolution). Perhaps we need to identify these rules and hard-code them into our systems.
I think we also need to better represent unbounded and flexible knowledge. In particular, I'm enthusiastic about the progress in representing graphs using neural networks, for example in the form of embeddings.
Stephen: Awesome! This is quite fresh to me and definitely something I hope myself and the readers check out after this interview.
I think it's time to ask questions based on your popular book. What prompted you to write the book "Hands-On Machine Learning with Scikit-Learn and TensorFlow" and why you think it is different from other ML books.
Aurélien: TensorFlow was open-sourced just before I left Google, but I had played with it a bit earlier. I really liked it, and I felt that it might become big. However, the documentation wasn't great, and as I was learning how to use it, I took notes to write a blog post. My notes started to become larger and larger, and I started to think about writing a book. I submitted the idea to O'Reilly, they liked it, and the adventure started. :)
I had read several Machine Learning books, and I enjoyed them, but most of them felt really theoretical, and the few that had some code were way too simple. I didn't feel that there was any book that really showed how to code neural networks professionally. This was actually a significant reason for writing the book in the first place.
Stephen: Beautiful! I must appreciate your efforts Aurélien. The book has helped a lot in my career and I am once again super grateful for it. I am sure it is the same as some readers too who perhaps may have found your book helpful as well.
I am curious, what are your favourite Machine Learning tools?
Aurélien: Scikit-Learn has the best documentation I've ever seen. It's beautifully designed, it's a wonderful example that many others followed. Keras too is really awesome, and they've done a great job integrating it with TensorFlow. I also like tf.data, and more generally TensorFlow 2.x. And obviously, PyTorch is great too. Colab is an amazing tool, and obviously Jupyter as well. Google Cloud's AutoML is very impressive.
Stephen: Aurélien thank you so much for such knowledgable and engaging responses in this section. The next section of questions is your advice to the readers.
Pause and Reflect
Why do you think it is important to adopt learning as a core ability if you want to succeed as a Data Scientist or Machine Learning Engineer?
Do you think it is important to join an AI community to grow your skills and give back? Will you join one based on Aurélien's recommendation? (If you have not.)
Let us know your thoughts in the comments at the end of the article.
Aurélien's Advice to You
Stephen: Aurélien, there is a lot going on in the field right now, how do you manage to keep track of the latest relevant happenings?
Aurélien: Lately, with the move to New Zealand and everything that's been happening, I have lost track of the latest developments. However, I like twitter to keep track of what's going on: if you follow the right people and companies, you're quickly aware of the latest hot thing. Then you can look into it in blogs or videos, and dig deeper by reading the paper. I also use arxiv sanity. And when reading papers, I take note of other papers I'd like to read, and I go down my list once in a while. When I was writing my book, I was reading perhaps 5 or 6 papers per day. But the rest of the time, it's more like a couple per week.
There's so much to learn, I think it's easy to feel overwhelmed. But the key is to have a good general understanding of the landscape and then dig deeper into your favorite topic. You can't be an expert in every single domain.
Stephen: Thank you for sharing. I must say, with most researchers and practitioners on Twitter these days, it's hard to not get up-to-date with the latest advancements especially if you follow the right people and the right feeds.
I'd love to know, how do you manage your work-life balance?
Aurélien: There are two aspects to work-life balance:
How do you feel about your work-life balance, and
How do your loved ones feel about the time you spend with them.
I'm really passionate about Machine Learning, so frankly work does not feel like work. When I was working on my book, I would literally spend entire days in front of my computer, researching topics, writing, coding, debugging, then starting over. And I was super happy! So the hard part (in my case) is more about making sure to spend enough quality time with my wife and children. And also (even more importantly), to make sure it's quality time.
While I was writing the book, it was sometimes a bit hard for my family, but apart from that period, work-life balance is really not a problem in my case.
Stephen: Thank you for being honest about the situation, Aurélien. I believe your insight here will be helpful to the readers.
One last question, Aurélien; what is one "golden" advice you will give beginners looking to become a world-class Machine Learning engineer and problem-solver?
Aurélien:
That's a tough question. I think joining a community, like you (Stephen) did, is probably the best advice I could give. Feeling supported and having someone to share your passion with is really important. Congrats to all of you for doing this!
I would also recommend making sure you follow a good methodology and you understand what exactly you're trying to achieve in each project (as I mentioned earlier).
Another important advice is to work by iterations. Don't go in a tunnel to build the perfect system, you might come out of it in 6 months. Instead, start with something super simple, like linear regression, then iterate.
Stephen: What a way to round up one of my best conversations this year! Thank you so much for these invaluable insights, Aurélien. We are grateful you agreed to share your knowledge and thoughts with us in such a profound manner. We hope you have a great time during your live session with us.
Well, that wraps it all up, readers! Thank you for reading this wonderful interview with Aurélien. We hope you learned a lot from it.
You can take action steps now by giving back to the community: Remember to share your thoughts with the community through the comment section below or on Twitter by sharing this article and tagging us @PHCSchoolOfAI along with points you learned.
Thank you once again for taking the time to enjoy this rather insightful Q and A with Aurélien. We will hopefully keep releasing sessions like these before each of the (#pmlcourse) classes with a guest speaker/facilitator. If you would like to ask Aurélien a question directly, feel free to reach him on Twitter or attend our YouTube Premiere event (info is on the e-flier at the start of the article) with him.
If you also enjoyed this Q and A with Aurélien, do leave a reaction to the story, hit the like button, and share it with your friends that may be interested. See you soon!