Think Less, Build More đŸ’ĄđŸƒđŸ»
All articles

February 17, 2019

How to Stay Up To Date as a Data/Research Scientist

How do I keep up with all the new research papers and new machine learning updates? Let me tell you what I do.

Views

  • career

Read the original article here.

1. CONFERENCE

At the Rework Deep Learning Conference

I want to start with conferences because I just came back from one in January 2019. This is probably the most expensive but also the most fun option in my opinion. Since I became a data scientist, I have attended the following conferences so far:

It is quite surreal to be in an environment full of experts, leaders, and people who are eager to learn the topics that you breathe and practice. Though I don’t find it easy going out of my way to talk to strangers at these conferences, everyone at the conference are happy to talk to you about their interests and projects. For example, I had many interesting conversations with people who sat next to me during lunch about their work and projects. It was really awesome learning about how different people and industries were applying deep learning.

Also, I really enjoyed attending presentations at the conference because you get to see what other people are working on and see the trends and new techniques. Seeing what research leaders like Google, Facebook, and OpenAI were up to is very motivating. Also, you get to hear what companies like Dropbox, eBay, Airbnb, Uber, and Netflix are up to with deep learning. After the conference, I reminded of the power of GAN (Generative Adversarial Network) and wanted to try learning PyTorch since version 1.0 was finally released in January 2019.

Since you are no longer in a structured school environment, I view these conferences as school classes. I remember when I attended the Spark Summit, I learned about Pandas UDF (on Spark version 2.3 I believe) and immediately started using it at work when I got back.

When you are learning things on your own or working on projects at your company, all the changes and trends in the industry may not be very tangible. But attending conferences really gives you the perspective and let you feel what everyone is up to in person.

There are a few ways to attend:

  • Have your company pay for it (best option). Ask for discounts if you are going as a group. Even if you don’t go as a group, they often provide discounts. So just ask!
  • Ask to volunteer at the conference and go for free (I have met someone who did this).
  • Attend as a student, and you can usually gets a steep discount.

Next, I want to try going to NIPS (Neural Information Processing Systems) and ICLR (International Conference on Learning Representations), which are more academically focused.

2. TWITTER

Example Deep Learning News on Twitter

I wasn’t really into twitter before, but I started it again because of deep learning news. I found out that many people (researchers and companies) tweet valuable and up-to-date information on twitter. As you see above, GoogleAI account very often posts updates on their deep learning research.

When I started following some of the famous deep learning people and company accounts (@GoogleAI, @OpenAI, @AndrewYNg, @KDNuggets, @Goodfellow_Ian, @YLeCun, @Karpathy), it was easy for me to find others because twitter kept recommending similar accounts.

My account is super boring at the moment, but feel free to follow: @jasjung_. I plan on using it to post spontaneous tips or updates, but mostly to learn from other deep learning stars.

3. ENGINEERING BLOGS AND EMAIL NEWSLETTERS

If you search for something like “best data science newsletter,” you will get a lot of great results. That is one of the ways I gathered resources.

3.1. Company Engineering/Technical Blogs

Image caption from Airbnb Tech Blog on Medium

I think of these engineering blogs as mini conferences that companies hold on paper to show off and share their latest and brightest achievements to the world. They very often have interesting experiments, research, and projects to show here. Because it is maintained by the company, the quality is usually very high. I think Medium started to play a big role here as many companies started to host their engineering and technical blogs on Medium. Here are a few sites that I enjoy!

Remember, the engineering blogs won’t only talk about machine learning but they often do and have many other interesting things to read about.

3.2. Newsletters

Apart from engineering blogs, there are many newsletters from online publications like Medium or individuals to which you can subscribe to. I think these contents are more personal and easier to absorb since anyone, like me, gets to write these articles. They often contain small, individual projects, whereas the projects from company blogs might be a bit far-reaching for individuals to actually try out.

  • Publications like Towards Data Science is definitely one of the ones I follow. I find daily newsletters to be too much, but a weekly subscription has been the right amount for me.
  • An individual newsletter I really enjoy is Machine Learning Is Fun. It sends out fun weekly articles that you can actually try out yourself if you wanted to.
  • machinelearningmastery.com by Jason Brownlee also contains many practical tutorials and codes.

I know there are a lot more out there, but I haven’t really explored too much yet. So if you know more, please let me know in the comments below!

4. RESEARCH PAPER

Last but not least, this is another free yet most difficult option. For this section, I recommend you check out the following post on Medium: “Getting started with reading Deep Learning Research papers: The Why and the How.” The author provides much useful information and tips on reading research papers. To summarize, if you are interested in reading machine learning research papers, you should go to Arxiv Sanity Preserver, a project by Andrej Karpathy. It basically shows you the most recent, popular research papers so you can read the most interesting ones. Using his own words to explain this project:

This project is a web interface that attempts to tame the overwhelming flood of papers on Arxiv. It allows researchers to keep track of recent papers, search for papers, sort papers by similarity to any paper, see recent popular papers, to add papers to a personal library, and to get personalized recommendations of (new or old) Arxiv papers


I said this is the most difficult option because I can’t just casually read these papers. I need to focus and need a longer time to read it. Sometimes, I need a piece of paper to go through the math. Even though this is the most difficult options for me, it is also the most rewarding as you really get to understand the topic well. I plan on having a Research Paper directory in my Machine Learning Github repository to document my readings. I will do my best to update it often.

If you need a paper to start with, check out the following paper. Many people regard this as one of the most influential papers in deep learning for its successful work in convolutional neural network.

Here is the abstract from that paper for your reference:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called “dropout” that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

5. YOUTUBE (2019–04–13 Update)

I am adding one more section here because I truly believe this is helpful! As many will agree, YouTube is a great place to learn for so many topics. I enjoy following Siraj Raval channel on YouTube. He posts many recent updates and tutorials on AI topics. They are pretty amusing to watch as well.

CONCLUSION

Staying up to date is a daunting task when the amount of new information is overwhelming. It is something I struggle with everyday. But I hope these resources can help you guide your journey of staying on top of it all. If you know any good resources, please share in the comment below! I would love to learn about them.

Lastly, don’t feel pressured to read and learn everything. It is probably impossible to learn everything so you have to pick and choose. Choose one that sparks joy for you. What’s the point of learning if you don’t enjoy the content?

Good luck and thank you for reading!

2019–07–31 Update. Check out my latest project www.Salary.Ninja and the corresponding article Welcome to Salary Ninja.

2019–10–30 Update. Check out my program (AlphaBlitz) that beats Facebook’s Word Blitz Game using deep learning. Like and subscribe! :)