Datasets

150+ Machine Learning Datasets

Introduction:

Without Data there is no Machine Learning, no AI, no Deep Learning. Because of heavy automation, IOT devices all around, there is no dirth of data. The first issue is, due to privacy and security related issues, data is not available for everyone. The second issue is cleaning this data. He third issues is getting complete data which can solve a given business problem. To get the complete data you need to get the data from multiple sources, identify the key to connect different records/sample of different sources. It is an expensive and time-consuming step of data science project. If you want to learn data science or want to solve any existing problem using new methods. Then you need some benchmarking framework in place which can display the model metrics (recall/precision/accuracy etc.) of each new approach (algorithm) against a given dataset. So datasets play a critical role in benchmarking algorithm performance.

Thus, machine learning rely heavily on high-quality datasets for training and evaluation. These datasets serve as the foundation for developing robust and accurate models across various AI domains, including classical machine learning, computer vision, natural language processing (NLP), audio processing, and time series analysis. Access to diverse and comprehensive datasets is crucial for researchers and practitioners to tackle real-world problems and advance the field of machine learning.

In this article, I am publishing a curated collection of datasets sourced from over 150 data sources. These datasets and data sources have been carefully selected to cover a wide range of domains, ensuring their relevance to different machine learning applications. Whether you are working on a NLP/text project, CV/image project, audio project, or time series forecasting, or classical machine learning, you’ll find valuable datasets to support your research and development efforts. Let’s dive into the world of machine learning datasets and discover the wealth of resources available to fuel your projects. If you dig each link, you will find hundreds, if not thousands of datasets under many of the links shared. I hope you will get benefitted from this work.

Note: These links I got from chrome bookmarks. At the time of writing this article, I have validated the link. If you find any link is not work / wrongly pointing / wrongly describing then please help me in improving this article. You can write to me at hari.prasad @ vedavit-ps .com.

Note: If you want to search image dataset on this page search “image”, for speech search “speech”

List of Datasets and Data Sources

Sno.	URL	Description
1.	100+ Interesting Data Sets for Statistics	100+ Interesting Data Sets for Statistics
2.	100+ Mammography Image Databases	Mammography Image Databases – 100 or more images of mammograms with ground truth. Additional images available by request, and links to several other mammography databases are provided. (Formats: homebrew)
3.	15 amazon datasets on data.world	amazon data on data.world	8 datasets available
4.	20 Free Big Data Sources	20 Free Big Data Sources
5.	332 Sport Datasets on data.world	sports data on data.world	338 datasets available
6.	4000+ Groningen Natural Image Database	Groningen Natural Image Database – 4000+ 1536×1024 (16 bit) calibrated outdoor images (Formats: homebrew)
7.	450+ UCI datasets
8.	538 Datasets	FiveThirtyEight: data and code related to their articles
9.	538 Datasets Summary Githu	FiveThirtyEight: data and code related to their articles
10.	57 products datasets on data.world
11.	622 UCI Archive Dataset	UCI Archive-Machine Learning Repository: Data Sets
12.	A Collective list of Free API for Datasets	A collective list of free APIs for use in software and web development.
13.	Academic Torrents- Large Research dataset	Academic Torrents: distributed network for sharing large research datasets
14.	The Air Freight data set is a ray-traced image sequence	Air Freight - The Air Freight data set is a ray-traced image sequence along with ground truth segmentation based on textural characteristics. (455 images + GT, each 160x120 pixels). (Formats: PNG)
15.	Air Freight Dataset - Computer Vision	Air Freight – The Air Freight data set is a ray-traced image sequence along with ground truth segmentation based on textural characteristics. (455 images + GT, each 160×120 pixels). (Formats: PNG)
16.	Allen Institutes Dataset	Datasets – Allen Institute for AI
17.	Amazon Datasets	Amazon Web Services Public Data Sets
18.	Amsterdam Library of Object Images - ALOI	Amsterdam Library of Object Images – ALOI is a color image collection of one-thousand small objects, recorded for scientific purposes. In order to capture the sensory variation in object recordings, we systematically varied viewing angle, illumination angle, and illumination color for each object, and additionally captured wide-baseline stereo images. We recorded over a hundred images of each object, yielding a total of 110,250 images for the collection. (Formats: png)
19.	Annotated face, hand, cardiac & meat images	Annotated face, hand, cardiac & meat images – Most images & annotations are supplemented by various ASM/AAM analyses using the AAM-API. (Formats: bmp,asf)
20.	Apigee	Apigee: explore dozens of popular APIs
21.	apilist.fun	API List: A public list of free APIs for programmers
22.	AT&T Laboratories Cambridge face database - Images	AT&T Laboratories Cambridge face database
23.	AVHRR Pathfinder	National Centre for Environment Information
24.	Awesome Deep Learning Database	Densely Sampled View Spheres – Densely sampled view spheres – upper half of the view sphere of two toy objects with 2500 images each. (Formats: tiff)
25.	Awesome Public Dataset
26.	Awesome Public Datasets	Awesome Public Datasets: Well-organized and frequently updated
27.	B2SHARE
28.	Berkeley Segmentation Dataset 500	Berkeley Segmentation Dataset 500
29.	Biometric Systems Lab	Biometric Systems Lab – University of Bologna
30.	Caltech Image Database	Caltech Image Database – about 20 images – mostly top-down views of small objects and toys. (Formats: GIF)
31.	CAVIAR video sequences of mall and public space behavior	CAVIAR video sequences of mall and public space behavior - 90K video frames in 90 sequences of various human activities, with XML ground truth of detection and behavior classification (Formats: MPEG2 & JPEG)
32.	CCITT Fax standard images	CCITT Fax standard images – 8 images (Formats: gif)
33.	Census of India
34.	Global Terrorism Database (GTD)
35.	CIFAR-10 and CIFAR-100	CIFAR-10 and CIFAR-100
36.	CMU CIL’s Stereo Data (Image)	CMU CIL’s Stereo Data with Ground Truth – 3 sets of 11 images, including color tiff images with spectroradiometry (Formats: gif, tiff)
37.	CMU PIE Database	CMU PIE Database - A database of 41,368 face images of 68 people captured under 13 poses, 43 illuminations conditions, and with 4 different expressions.
38.	CMU VASC Image Database	CMU VASC Image Database – Images, sequences, stereo pairs (thousands of images) (Formats: Sun Rasterimage)
39.	College Scorecard Data
40.	Columbia-Utrecht Reflectance and Texture Database	Columbia-Utrecht Reflectance and Texture Database – Texture and reflectance measurements for over 60 samples of 3D texture, observed with over 200 different combinations of viewing and illumination directions. (Formats: bmp)
41.	Computational Colour Constancy Data	Computational Colour Constancy Data - A dataset oriented towards computational color constancy, but useful for computer vision in general. It includes synthetic data, camera sensor data, and over 700 images. (Formats: tiff)
42.	Computational Vision Lab	Computational Vision Lab
43.	Content-based image retrieval database	Content-based image retrieval database - 11 sets of color images for testing algorithms for content-based retrieval. Most sets have a description file with names of objects in each image. (Formats: jpg)
44.	Cricket Data
45.	Crowdanalytics
46.	Crowdflower Dataset	CrowdFlower: interesting datasets created or enhanced by their contributors
47.	Crunchbase	Crunchbase: Discover innovative companies and the people behind them
48.	CVD Foundation Open Images	Open Images dataset – Open Images is a dataset of ~9 million URLs to images that have been annotated with labels spanning over 6000 categories.
49.	Data world
50.	Data.lacity.org	DataLA
51.	DataCamp
52.	DataInnovation Dataset Blog	Center for Data Innovation: blog posts about interesting, recently-released data sets.
53.	Dataverse.org	Dataverse Project: searchable archive of research data
54.	DC Open Data Catalog	DC Open Data Catalog / OpenDataDC
55.	Deep Fashion - Images	Large-scale Fashion (DeepFashion) Database – Contains over 800,000 diverse fashion images. Each image in this dataset is labeled with 50 categories, 1,000 descriptive attributes, bounding box and clothing landmarks
56.	Devanagari Handwritten Character Dataset - Images
57.	Donor Choose	Donors Choose: data related to their projects
58.	Face and Gesture images and image sequences	Face and Gesture images and image sequences – Several image datasets of faces and gestures that are ground truth annotated for benchmarking http://www.fg-net.org/
59.	FG-NET Facial Aging Database	FG-NET Facial Aging Database – Database contains 1002 face images showing subjects at different ages. (Formats: jpg)
60.	Finding Datasets from inside-r.org	inside-R
61.	Flickr 30k, Images with Caption	Flickr 30k
62.	Flickr 8k, Images with Caption	Flickr 8k
63.	Flickr Data 100 Million Yahoo dataset, Images	Flickr Data 100 Million Yahoo dataset
64.	FVC2000 Fingerprint Databases, Images	FVC2000 Fingerprint Databases - FVC2000 is the First International Competition for Fingerprint Verification Algorithms. Four fingerprint databases constitute the FVC2000 benchmark (3520 fingerprints in all).
65.	Gapminder Data
66.	German Fingerspelling Database	German Fingerspelling Database – The database contains 35 gestures and consists of 1400 image sequences that contain gestures of 20 different persons recorded under non-uniform daylight lighting conditions. http://www-i6.informatik.rwth-aachen.de/~dreuw/database.html
67.	Getting Stock Data
68.	Github-DataMeet	Datameet is a community of Data Science enthusiasts.
69.	Google House Numbers from street view
70.	Google Scholar
71.	HowStat	HowSTAT! The Cricket Statisticians – Home Page
72.	github.com/TheUpShot	The Upshot: data related to their articles
73.	Huggingface datasets
74.	Humanitarian Data Exchange	Humanitarian Data Exchange
75.	IEEE DataPort	Data Competitions	IEEE DataPort
76.	IEEN Image Library	IEN Image Library – 1000+ images, mostly outdoor sequences (Formats: raw, ppm)
77.	Image Analysis Laboratory	Image Analysis Laboratory – Images obtained from a variety of imaging modalities — raw CFA images, range images and a host of “medical images”. (Formats: homebrew)
78.	Image QA	Image QA
79.	ImageNet	ImageNet
80.	IMDb Top 250 Movies	Ratings and Reviews for New Movies and TV Shows – IMDb
81.	IMF-Exchange Rate	IMF-Exchange Rate Archives by Month
82.	Indian Govt
83.	Indian Liver Patient Dataset
84.	INRIA	INRIA
85.	Institute of Computer Graphics and Vision	Institute of Computer Graphics and Vision
86.	Inter University Consortium for Politics & Social	Inter-university Consortium for Political and Social Research
87.	Kaggle Datasets	Kaggle provides datasets with their challenges, but each competition has its own rules as to whether the data can be used outside of the scope of the competition.
88.	kdnuggets	kdnuggets- Datasets for Data Mining and Data Science
89.	Mammography Image Databases	Mammography Image Databases - 100 or more images of mammograms with ground truth. Additional images available by request, and links to several other mammography databases are provided. (Formats: homebrew)
90.	Mashape - Explore APIs	Mashape: explore hundreds of APIs
91.	Microsoft COCO	Microsoft COCO
92.	Million Song Dataset	Million Song Dataset
93.	MIT Vision Texure	MIT Vision Texture – Image archive (100+ images) (Formats: ppm)
94.	MNIST Handwritten digits	MNIST Handwritten digits
95.	NLM HyperDoc Visible Human Project	NLM HyperDoc Visible Human Project - Color, CAT and MRI image samples - over 30 images (Formats: jpeg)
96.	NYC Open Data socrata	NYC Open Data
97.	OASIS 1	OASIS-1 (Open Access Series of Imaging Studies)
98.	OASIS Brain - Imaging Studies	Cross-Sectional MRI Data in Young, Middle Aged, Nondemented, and Demented Older Adults
99.	Open Images is a dataset of ~9 million URLs	Open Images dataset - Open Images is a dataset of ~9 million URLs to images that have been annotated with labels spanning over 6000 categories.
100.	Photometric 3D Surface Texture Database	Photometric 3D Surface Texture Database - This is the first 3D texture database which provides both full real surface rotations and registered photometric stereo data (30 textures, 1680 images). (Formats: TIFF)
101.	Pittsburgh Science of Learning	Pittsburgh Science of Learning Center’s DataShop
102.	ProPublica Data Store	ProPublica Data Store
103.	Python API for Datasets	Python APIs: Python wrappers for many APIs
104.	Quanddl	Quandl: over 10 million financial, economic, and social datasets
105.	R Datasets	Rdatasets: collection of 700+ datasets originally distributed with R packages
106.	RapidAPI.com	25 Free Public APIs for Developers & Free Alternatives List
107.	rdatamining.com	RDataMining.com
108.	Reddit Dataset	Datasets subreddit: ask for help finding a specific data set, or post your own
109.	Reddit Dataset from 2500 subreddits	Reddit Top 2.5 Million: all-time top 1,000 posts from each of the top 2,500 subreddits
110.	Reddit Dataset Jeopardy Question	200,000+ Jeopardy questions
111.	research.yahoo.com
112.	Sebastian Raschka	Sebastian Raschka: datasets categorized by format and topic
113.	Smartcities Data Govt of India
114.	Stanford Edu Dataset	Stanford Large Network Dataset Collection: graph data
115.	Suicide Rates 1985-2013	Suicide Rates Overview 1985 to 2016	Kaggle
116.	Sunlight Foundation Govt Data	Sunlight Foundation: government-focused data
117.	India, Surat City
118.	UP Govt Economics	Directorate of Economics and Statistics UP Govt.
119.	UP Smart Cities
120.	Tamilnadu	37K Resources, 4,134 Catalog, 101 Departments
121.	The MIT-CSAIL Database of Objects and Scenes	The MIT-CSAIL Database of Objects and Scenes - Database for testing multiclass object detection and scene recognition algorithms. Over 72,000 images with 2873 annotated frames. More than 50 annotated object classes. (Formats: jpg)
122.	Tiny Images 80 Million tiny images	Tiny Images 80 Million tiny images6.
123.	Traffic Image Sequences and ‘Marbled Block’ Sequence	Traffic Image Sequences and ‘Marbled Block’ Sequence - thousands of frames of digitized traffic image sequences as well as the ‘Marbled Block’ sequence (grayscale images) (Formats: GIF)
124.	U Oulu wood and knots database	U Oulu wood and knots database - Includes classifications - 1000+ color images (Formats: ppm)
125.	UC Irvine Machine Learning Repository	UC Irvine Machine Learning Repository
126.	UCI	UC Irvine Machine Learning Repository: datasets specifically designed for machine learning
127.	UCI Archive 620+ datasets
128.	UCI-Liver Disorder Datasets	UCI Machine Learning Repository: Liver Disorders Data Set
129.	UFO - Geolocation and Time Dataset	UFO reports: geolocated and time-standardized UFO reports for close to a century
130.	UK Govt	data.gov.uk
131.	University of Oulu Physics-based Face Database	University of Oulu Physics-based Face Database - contains color images of faces under different illuminants and camera calibration conditions as well as skin spectral reflectance measurements of each person.
132.	University of Oulu Texture Database	University of Oulu Texture Database - Database of 320 surface textures, each captured under three illuminants, six spatial resolutions and nine rotation angles. A set of test suites is also provided so that texture segmentation, classification, and retrieval algorithms can be tested in a standard manner. (Formats: bmp, ras, xv)
133.	US Census Bureau	US Census Bureau
134.	US Gov 256K datasets	The Home of the U.S. Government’s Open Data
135.	US Govt	data.gov (see also: Project Open Data Dashboard)
136.	US Students Univerties
137.	USF Range Image Data with Segmentation	USF Range Image Data with Segmentation Ground Truth - 80 image sets (Formats: Sun rasterimage)
138.	Vanderbilt edu dataset websites
139.	Vanderbilt edu datasets
140.	VQA	Visual Question Answering
141.	Wikipedia Dataset	Wikipedia:Database download – Wikipedia
142.	Wiry Object Recognition Database	Wiry Object Recognition Database - Thousands of images of a cart, ladder, stool, bicycle, chairs, and cluttered scenes with ground truth labelings of edges and regions.
143.	World Bank Open Data	World Bank Open Data
144.	Yale Face Database - 165 images	Yale Face Database - 165 images (15 individuals) with different lighting, expression, and occlusion configurations.
145.	Yale Face Database B - 5760	Yale Face Database B - 5760 single light source images of 10 subjects each seen under 576 viewing conditions (9 poses x 64 illumination conditions). (Formats: PGM)
146.	Yelp.com Datasets Challenge	Yelp Dataset Challenge: Yelp reviews, business attributes, users, and more from 10 cities
147.	YouTube-8M Dataset	YouTube-8M Dataset - YouTube-8M is a large-scale labeled video dataset that consists of 8 million YouTube video IDs and associated labels from a diverse vocabulary of 4800 visual entities.
148.	Aylien News Data API
149.	Aylien Datasets
150.	Finance Datasets on Kaggle
151.	Github philipperemy/financial-news-dataset
152.	FT Markets Data
153.	IMF Datasets
154.	Worldbank Datasets
155.	Reuter Finance News Dataset Title Only
156.	Bloomberg + Reuter Finance News Dataset
157.	Enron Email Dataset	It has more than 500K emails of over 150 users. The size of the data is around 432Mb. Out of 150 users, most of the users are the senior management of Enron.
158.	Chatbot Intents Dataset	The dataset for a chatbot is a JSON file that has disparate tags like goodbye, greetings, pharmacy_search, hospital_search, etc. Every tag has a list of patterns that a user can ask, and the chatbot will respond according to that pattern. The dataset is perfect for understanding how chatbot data works.
159.	Parkinson Dataset	Parkinson dataset contains biomedical measurements, 195 records of people with 23 different attributes. This data is used to differentiate healthy people and people with Parkinson’s disease.
160.	Mall Customers Dataset	The Mall customers dataset holds the details about people visiting the mall. The dataset has an age, customer id, gender, annual income, and spending score. It gains insights from the data and divides the customers into different groups based on their behaviors.
161.	Google Trends Data Portal	Google trends data can be used to examine and analyze the data visually. We can find out what’s trending and what people are searching for.
162.	Recommender Systems and Personalization Datasets	This is a portal to a collection of rich datasets that were used in lab research projects at UCSD. It contains various datasets from popular websites like Goodreads book reviews, Amazon product reviews, bartending data, data from social media, etc that are used in building a recommender system.
163.	GTSRB (German traffic sign recognition benchmark) Dataset	Build a model using a deep learning framework that classifies traffic signs and also recognizes the bounding box of signs. The traffic sign classification is also useful in autonomous vehicles for identifying signs and then taking appropriate actions.
164.	Cityscapes Dataset	It contains high-quality pixel-level annotations of video sequences taken in 50 different city streets. The dataset is useful in semantic segmentation and training deep neural networks to understand the urban scene.
165.	Kinetics Dataset	There are three different datasets for Kinetics: Kinetics 400, Kinetics 600, and Kinetics 700 dataset. This is a large scale dataset that contains a URL link to around 6.5 million high-quality videos. Build a human action recognition model and detect the action of a human.
166.	IMDB-Wiki dataset	The IMDB-Wiki dataset is one of the largest open-source datasets for face images with labeled gender and age. The images are collected from IMDB and Wikipedia. It has 5 million-plus labeled images.
167.	Color Detection Dataset	The dataset contains a CSV file that has 865 color names with their corresponding RGB (red, green, and blue) values of the color.
168.	Libri Speech Dataset	This dataset contains a large number of English speeches that are derived from the LibriVox project. It has 1000 hours of English-read speech in various accents. The objective of speech recognition is to automatically identify what is being said in the audio.
169.	Breast Histopathology Images Dataset	This dataset contains 2,77,524 images of size 50×50 extracted from 162 mount slide images of breast cancer specimens scanned at 40x. There are 1,98,738 negative tests and 78,786 positive tests with IDC.
170.	youtube-8M analytics
171.	Temporal concept localization within video - YouTube-8M, Link2	The YouTube-8M Segments dataset is an extension of the YouTube-8M dataset with human-verified segment annotations. In addition to annotating videos, we would like to temporally localize the entities in the videos, i.e., find out when the entities occur.
172.	CodaLab	Hundreds of interesting datasets.
173.	Hate Speech Dataset in Devnagari from Kaggle
174.	Stanford Speech Dataset
175.	TED-LIUM corpus release 3
176.	40 Open Source Audio Datasets
177.	Microsoft Datasets
178.	9 Voice Datasets from cmwire
179.	BBC Datasets	Consists of 2225 documents from the BBC news website corresponding to stories in five topical areas from 2004-2005. Class Labels: 5 (business, entertainment, politics, sport, tech)

Conclusion:

Machine learning datasets play a pivotal role in the development and advancement of various machine learning applications. In this article, we have explored an extensive collection of datasets obtained from more than 150 data sources, encompassing classical machine learning, computer vision, NLP/NLU, audio processing, and time series analysis.

By leveraging these diverse datasets, researchers and practitioners can build more robust and accurate machine learning models. These datasets provide the necessary ingredients for training, testing, and validating models across different domains, enabling the development of intelligent systems that can understand, interpret, and make predictions from complex data.

As the field of machine learning continues to evolve, the availability of high-quality datasets remains crucial. Whether you are embarking on a new project or seeking to enhance your existing models, exploring and utilizing these curated datasets will empower you to push the boundaries of what is possible in machine learning.

Remember, the power of machine learning lies not only in the algorithms and techniques but also in the data that fuels them. Embrace the vast array of datasets at your disposal and embark on exciting journeys of discovery and innovation in the world of machine learning.