Laboratory for experimental technologies

  • by Richie Bachala
    As data continues to grow in importance and become more complex, the need for skilled data engineers has never been greater. But what is data engineering, and why is it so important? In this blog post, we will discuss the essential components of a functioning data engineering practice and why data engineering is becoming increasingly […] The post Building a Data Engineering Center of Excellence appeared first on Towards Data Science.
  • by David Martin
    In this fifth part of my series, I will outline the steps for creating a Docker container for training your image classification model, evaluating performance, and preparing for deployment. AI/ML engineers would prefer to focus on model training and data engineering, but the reality is that we also need to understand the infrastructure and mechanics […] The post Learnings from a Machine Learning Engineer — Part 5: The Training appeared first on Towards Data Science.
  • by David Martin
    In this third part of my series, I will explore the evaluation process which is a critical piece that will lead to a cleaner data set and elevate your model performance. We will see the difference between evaluation of a trained model (one not yet in production), and evaluation of a deployed model (one making real-world predictions). In Part 1, […] The post Learnings from a Machine Learning Engineer — Part 3: The Evaluation appeared first on Towards Data Science.
  • by David Martin
    It is said that in order for a machine learning model to be successful, you need to have good data. While this is true (and pretty much obvious), it is extremely difficult to define, build, and sustain good data. Let me share with you the unique processes that I have learned over several years building […] The post Learnings from a Machine Learning Engineer — Part 1: The Data appeared first on Towards Data Science.
  • by David Martin
    In this latest part of my series, I will share what I have learned on selecting a model for image classification and how to fine tune that model. I will also show how you can leverage the model to accelerate your labelling process, and finally how to justify your efforts by generating usage and performance […] The post Learnings from a Machine Learning Engineer — Part 4: The Model appeared first on Towards Data Science.
  • by David Martin
    In Part 1, we discussed the importance of collecting good image data and assigning proper labels for your image classification project to be successful. Also, we talked about classes and sub-classes of your data. These may seem pretty straight forward concepts, but it’s important to have a solid understanding going forward. So, if you haven’t, please […] The post Learnings from a Machine Learning Engineer — Part 2: The Data Sets appeared first on Towards Data Science.
  • by Sara A. Metwalli
    I am sure the quantum hype has reached every person in tech (and outside it, most probably). With some over-the-top claims, like “some company has proved quantum supremacy,” “the quantum revolution is here,” or my favorite, “quantum computers are here, and it will make classical computers obsolete.” I am going to be honest with you; […] The post Should Data Scientists Care About Quantum Computing? appeared first on Towards Data Science.
  • by Mahmoud Abdelaziz
    Let’s say you are in a customer care center, and you would like to know the probability distribution of the number of calls per minute, or in other words, you want to answer the question: what is the probability of receiving zero, one, two, … etc., calls per minute? You need this distribution in order […] The post Method of Moments Estimation with Python Code appeared first on Towards Data Science.
  • by Umair Ali Khan
    The basic principle of Large Language Models (LLMs) is very simple: to predict the next word (or token) in a sequence of words based on statistical patterns in their training data. However, this seemingly simple capability turns out to be incredibly sophisticated when it can do a number of amazing tasks such as text summarization, […] The post How to Measure the Reliability of a Large Language Model’s Response appeared first on Towards Data Science.