The following tables compare notable software frameworks, libraries, and computer programs for deep learning applications.
Deep learning software by name
| Software | Creator | Initial release | Software licensea | Open source
|
Platform | Written in | Interface | OpenMP support | OpenCL support | CUDA support | Automatic differentiation2 | Has pretrained models | Parallel execution
(multi node) |
Actively developed
| ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BigDL | Jason Dai (Intel) | 2016 | Apache 2.0 | Yes | Apache Spark | Scala | Scala, Python | No | No | Yes | Yes | Yes | Yes | |||||
| Caffe | Berkeley Vision and Learning Center | 2013 | BSD | Yes | Linux, macOS, Windows3 | C++ | Python, MATLAB, C++ | Yes | Under development4 | Yes | No | Yes | Yes5 | Yes | Yes | No | ? | No6 |
| Chainer | Preferred Networks | 2015 | BSD | Yes | Linux, macOS | Python | Python | No | No | Yes | No | Yes | Yes | Yes | Yes | No | Yes | No7 |
| Deeplearning4j | Skymind engineering team; Deeplearning4j community; originally Adam Gibson | 2014 | Apache 2.0 | Yes | Linux, macOS, Windows, Android (Cross-platform) | C++, Java | Java, Scala, Clojure, Python (Keras), Kotlin | Yes | No8 | Yes910 | No | Computational Graph | Yes11 | Yes | Yes | Yes | Yes12 | Yes |
| DeepSpeed | Microsoft | 2019 | Apache 2.0 | Yes | Linux, macOS, Windows | Python, C++, CUDA | Python | No | No | Yes | No | Yes | Yes | Yes | Yes | No | Yes | Yes |
| Dlib | Davis King | 2002 | Boost Software License | Yes | Cross-platform | C++ | C++, Python | Yes | No | Yes | No | Yes | Yes | No | Yes | Yes | Yes | Yes |
| Fastai | fast.ai | 2018 | Apache 2.0 | Yes | Linux, macOS, Windows | Python, CUDA | Python | No | No | Yes | No | Yes | Yes | Yes | Yes | No | Yes | Yes |
| Flux | Mike Innes | 2017 | MIT | Yes | Linux, macOS, Windows (Cross-platform) | Julia | Julia | Yes | No | Yes | Yes13 | Yes | Yes | No | Yes | Yes | ||
| Horovod | Uber Technologies | 2017 | Apache 2.0 | Yes | Linux, macOS, Windows | Python, C++, CUDA | Python | No | No | Yes | No | Yes | Yes | Yes | Yes | No | Yes | Yes |
| Intel Data Analytics Acceleration Library | Intel | 2015 | Apache 2.0 | Yes | Linux, macOS, Windows on Intel CPU14 | C++, Python, Java | C++, Python, Java14 | Yes | No | No | No | Yes | No | Yes | Yes | Yes | ||
| Intel Math Kernel Library 2017 15 and later | Intel | 2017 | Proprietary | No | Linux, macOS, Windows on Intel CPU16 | C/C++, DPC++, Fortran | C17 | Yes18 | No | No | No | Yes | No | Yes19 | Yes19 | No | Yes | |
| Google JAX | 2018 | Apache 2.0 | Yes | Linux, macOS, Windows | Python | Python | Only on Linux | No | Yes | No | Yes | Yes | ||||||
| Keras | François Chollet | 2015 | MIT | Yes | Linux, macOS, Windows | Python | Python, R | Only if using Theano as backend | Can use Theano, Tensorflow or PlaidML as backends | Yes | No | Yes | Yes20 | Yes | Yes | No21 | Yes22 | Yes |
| MATLAB + Deep Learning Toolbox (formerly Neural Network Toolbox) | MathWorks | 1992 | Proprietary | No | Linux, macOS, Windows | C, C++, Java, MATLAB | MATLAB | No | No | Train with Parallel Computing Toolbox and generate CUDA code with GPU Coder23 | No | Yes24 | Yes2526 | Yes25 | Yes25 | Yes | With Parallel Computing Toolbox27 | Yes |
| Microsoft Cognitive Toolkit (CNTK) | Microsoft Research | 2016 | MIT28 | Yes | Windows, Linux29 (macOS via Docker on roadmap) | C++ | Python (Keras), C++, Command line,30 BrainScript31 (.NET on roadmap32) | Yes33 | No | Yes | No | Yes | Yes34 | Yes35 | Yes35 | No36 | Yes37 | No38 |
| MindSpore | Huawei | 2020 | Apache 2.0 | Yes | Linux, Windows, macOS, EulerOS, openEuler, OpenHarmony, Oniro OS, HarmonyOS, Android | C++, Rust, Julia, Python, ArkTS, Cangjie, Java (Lite) | ||||||||||||
| ML.NET | Microsoft | 2018 | MIT | Yes | Windows, Linux, macOS | C#, C++ | C#, F# | Yes | ||||||||||
| Apache MXNet | Apache Software Foundation | 2015 | Apache 2.0 | Yes | Linux, macOS, Windows,3940 AWS, Android,41 iOS, JavaScript42 | Small C++ core library | C++, Python, Julia, MATLAB, JavaScript, Go, R, Scala, Perl, Clojure | Yes | No | Yes | No | Yes43 | Yes44 | Yes | Yes | Yes | Yes45 | No |
| Neural Designer | Artelnics | 2014 | Proprietary | No | Linux, macOS, Windows | C++ | Graphical user interface | Yes | No | Yes | No | Analytical differentiation | No | No | No | No | Yes | Yes |
| OpenNN | Artelnics | 2003 | GNU LGPL | Yes | Cross-platform | C++ | C++ | Yes | No | Yes | No | ? | Yes46 | No | No | No | ? | Yes |
| PlaidML | Vertex.AI, Intel | 2017 | Apache 2.0 | Yes | Linux, macOS, Windows | Python, C++, OpenCL | Python, C++ | ? | Some OpenCL ICDs are not recognized | No | No | Yes | Yes | Yes | Yes | Yes | Yes | |
| PyTorch | Meta AI | 2016 | BSD | Yes | Linux, macOS, Windows, Android47 | Python, C, C++, CUDA | Python, C++, Julia, R48 | Yes | Via separately maintained package495051 | Yes | Yes | Yes | Yes | Yes | Yes | Yes52 | Yes | Yes |
| PyTorch Lightning | Lightning-AI (originally William Falcon)53 | 2019 | Apache 2.0 | Yes | Linux, macOS, Windows | Python | Python | Yes | Via PyTorch | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes54 | Yes |
| Apache SINGA | Apache Software Foundation | 2015 | Apache 2.0 | Yes | Linux, macOS, Windows | C++ | Python, C++, Java | No | Supported in V1.0 | Yes | No | ? | Yes | Yes | Yes | Yes | Yes | Yes |
| TensorFlow | Google Brain | 2015 | Apache 2.0 | Yes | Linux, macOS, Windows,5556 Android | C++, Python, CUDA | Python (Keras), C/C++, Java, Go, JavaScript, R,57 Julia, Swift | No | On roadmap58 but already with SYCL59 support | Yes | Yes | Yes60 | Yes61 | Yes | Yes | Yes | Yes | Yes |
| TensorRT | Nvidia | 2017 | Proprietary | No | Linux, Windows on Nvidia GPUs | C++, Python, CUDA | C++, Python | No | No | Yes | No | No | No | No | Yes | No | No | Yes |
| Theano | Université de Montréal | 2007 | BSD | Yes | Cross-platform | Python | Python (Keras) | Yes | Under development62 | Yes | No | Yes6364 | Through Lasagne's model zoo65 | Yes | Yes | Yes | Yes66 | No |
| Torch | Ronan Collobert, Koray Kavukcuoglu, Clement Farabet | 2002 | BSD | Yes | Linux, macOS, Windows,67 Android,68 iOS | C, Lua | Lua, LuaJIT,69 C, utility library for C++/OpenCL70 | Yes | Third party implementations7172 | Yes7374 | No | Through Twitter's Autograd75 | Yes76 | Yes | Yes | Yes | Yes67 | No |
| RANT (Real-time Artificial Neural Tool)77 | Douglas Santry | 2023 | BSD | Yes | Linux, macOS, Windows | C++, Python | C++, Python | No | No | No | No | No | Yes | No | Yes | No | No | Yes |
| Wolfram Mathematica 1078 and later | Wolfram Research | 2014 | Proprietary | No | Windows, macOS, Linux, Cloud computing | C++, Wolfram Language, CUDA | Wolfram Language | Yes | No | Yes | No | Yes | Yes79 | Yes | Yes | Yes | Yes80 | Yes |
| Software | Creator | Initial release | Software licensea | Open source
|
Platform | Written in | Interface | OpenMP support | OpenCL support | CUDA support | Automatic differentiation2 | Has pretrained models | Parallel execution
(multi node) |
Actively developed
|
- Licenses here are a summary, and are not taken to be complete statements of the licenses. Some libraries may use other libraries internally under different licenses
Comparison of machine learning model compatibility
| Format name | Design goal | Compatible with other formats | Self-contained DNN Model | Pre-processing and Post-processing | Run-time configuration for tuning & calibration | DNN model interconnect | Common platform |
|---|---|---|---|---|---|---|---|
| TensorFlow, Keras, Caffe, Torch | Algorithm training | No | No / Separate files in most formats | No | No | No | Yes |
| ONNX | Algorithm training | Yes | No / Separate files in most formats | No | No | No | Yes |
See also
See also
- Comparison of machine learning software
- Comparison of statistical packages
- Comparison of cognitive architectures
- Lists of open-source artificial intelligence software
- List of datasets for machine-learning research
- List of numerical-analysis software
- MLIR compiler — sub-project of the LLVM designed for machine learning, hardware acceleration, and high-level synthesis.
- tinygrad — being developed by George Hotz
References
References
- "Deep Learning — ROCm 4.5.0 documentation". Archived from the original on 2022-12-05. Retrieved 2022-09-27.
- Atilim Gunes Baydin; Barak A. Pearlmutter; Alexey Andreyevich Radul; Jeffrey Mark Siskind (20 February 2015). "Automatic differentiation in machine learning: a survey". arXiv:1502.05767 [cs.LG].
- "Microsoft/caffe". GitHub. 30 October 2021.
- "Caffe: a fast open framework for deep learning". GitHub. July 19, 2019.
- "Caffe | Model Zoo". caffe.berkeleyvision.org.
- GitHub - BVLC/caffe: Caffe: a fast open framework for deep learning., Berkeley Vision and Learning Center, 2019-09-25, retrieved 2019-09-25
- Preferred Networks Migrates its Deep Learning Research Platform to PyTorch, 2019-12-05, retrieved 2019-12-27
- "Support for Open CL: Issue #27: deeplearning4j/nd4j". GitHub.
- "N-Dimensional Scientific Computing for Java". Archived from the original on 2016-10-16. Retrieved 2016-02-05.
- "Comparing Top Deep Learning Frameworks". Deeplearning4j. Archived from the original on 2017-11-07. Retrieved 2017-10-31.
- Chris Nicholson; Adam Gibson. "Deeplearning4j Models". Archived from the original on 2017-02-11. Retrieved 2016-03-02.
- Deeplearning4j. "Deeplearning4j on Spark". Deeplearning4j. Archived from the original on 2017-07-13. Retrieved 2016-09-01.
{{cite web}}: CS1 maint: numeric names: authors list (link) - "Metalhead". FluxML. 29 October 2021.
- "Intel® Data Analytics Acceleration Library (Intel® DAAL)". software.intel.com. November 20, 2018.
- "Intel® Math Kernel Library Release Notes and New Features". Intel.
- "Intel® Math Kernel Library (Intel® MKL)". software.intel.com. September 11, 2018.
- "Deep Neural Network Functions". software.intel.com. May 24, 2019.
- "Using Intel® MKL with Threaded Applications". software.intel.com. June 1, 2017.
- "Intel® Xeon Phi™ Delivers Competitive Performance For Deep Learning—And Getting Better Fast". software.intel.com. March 21, 2019.
- "Applications - Keras Documentation". keras.io.
- "Is there RBM in Keras?: Issue #461: keras-team/keras". GitHub.
- "Does Keras support using multiple GPUs?: Issue #2436: keras-team/keras". GitHub.
- "GPU Coder - MATLAB & Simulink". MathWorks. Retrieved 13 November 2017.
- "Automatic Differentiation Background - MATLAB & Simulink". MathWorks. September 3, 2019. Retrieved November 19, 2019.
- "Neural Network Toolbox - MATLAB". MathWorks. Retrieved 13 November 2017.
- "Deep Learning Models - MATLAB & Simulink". MathWorks. Retrieved 13 November 2017.
- "Parallel Computing Toolbox - MATLAB". MathWorks. Retrieved 13 November 2017.
- "CNTK/LICENSE.md at master: Microsoft/CNTK". GitHub.
- "Setup CNTK on your machine". GitHub.
- "CNTK usage overview". GitHub.
- "BrainScript Network Builder". GitHub.
- ".NET Support: Issue #960: Microsoft/CNTK". GitHub.
- "How to train a model using multiple machines?: Issue #59: Microsoft/CNTK". GitHub.
- "Prebuilt models for image classification: Issue #140: microsoft/CNTK". GitHub.
- "CNTK - Computational Network Toolkit". Microsoft Corporation.
- "Restricted Boltzmann Machine with CNTK #534". GitHub, Inc. 27 May 2016. Retrieved 30 October 2023.
- "Multiple GPUs and machines". Microsoft Corporation.
- "Disclaimer". CNTK TEAM. 6 November 2021.
- "Releases: dmlc/mxnet". GitHub.
- "Installation Guide — mxnet documentation". Readthdocs.
- "MXNet Smart Device". ReadTheDocs. Archived from the original on 2016-09-21. Retrieved 2016-05-19.
- "MXNet.js". GitHub. 28 October 2021.
- "— Redirecting to mxnet.io". mxnet.readthedocs.io.
- "Model Gallery". GitHub. 29 October 2022.
- "Run MXNet on Multiple CPU/GPUs with Data Parallel". GitHub.
- "Model Types". 31 August 2023.
- "PyTorch". Dec 17, 2021.
- "Falbel D, Luraschi J (2023). torch: Tensors and Neural Networks with 'GPU' Acceleration". torch.mlverse.org. Retrieved 2023-11-28.
- "OpenCL build of pytorch: (in-progress, not useable) - hughperkins/pytorch-coriander". GitHub. July 14, 2019.
- "DLPrimitives/OpenCL out of tree backend for pytorch - artyom-beilis/pytorch_dlprim". GitHub. Jan 21, 2022.
- "OpenCL Support: Issue #488: pytorch/pytorch". GitHub.
- "Restricted Boltzmann Machines (RBMs) in PyTorch". GitHub. 14 November 2022.
- https://github.com/williamFalcon
- https://lightning.ai/docs/litserve/how-tos/parallelism-types
- "Install TensorFlow with pip".
- "TensorFlow 0.12 adds support for Windows".
- Allaire, J.J.; Kalinowski, T.; Falbel, D.; Eddelbuettel, D.; Yuan, T.; Golding, N. (28 September 2023). "tensorflow: R Interface to 'TensorFlow'". The Comprehensive R Archive Network. Retrieved 30 October 2023.
- "tensorflow/roadmap.md at master". GitHub. January 23, 2017. Retrieved May 21, 2017.
- "OpenCL support". GitHub.
- "TensorFlow". TensorFlow.
- "Models and examples built with TensorFlow". GitHub. July 19, 2019.
- "Using the GPU: Theano 0.8.2 documentation". Archived from the original on 2017-04-01. Retrieved 2016-01-21.
- "gradient – Symbolic Differentiation — Theano 1.0.0 documentation". deeplearning.net.
- "Automatic vs. Symbolic differentiation".
- "Recipes/modelzoo at master: Lasagne/Recipes". GitHub.
- "Using multiple GPUs — Theano 1.0.0 documentation". deeplearning.net.
- "torch/torch7". GitHub. July 18, 2019.
- "GitHub - soumith/torch-android: Torch-7 for Android". GitHub. 13 October 2021.
- "Torch7: A MATLAB-like Environment for Machine Learning" (PDF).
- "GitHub - jonathantompson/jtorch: An OpenCL Torch Utility Library". GitHub. 18 November 2020.
- "Cheatsheet". GitHub.
- "cltorch". GitHub.
- "Torch CUDA backend". GitHub.
- "Torch CUDA backend for nn". GitHub.
- "Autograd automatically differentiates native Torch code: twitter/torch-autograd". GitHub. July 9, 2019.
- "ModelZoo". GitHub.
- "RANT".
- "Launching Mathematica 10". Wolfram.
- "Wolfram Neural Net Repository of Neural Network Models". resources.wolframcloud.com.
- "Parallel Computing—Wolfram Language Documentation". reference.wolfram.com.
- "Deep Learning — ROCm 4.5.0 documentation". Archived from the original on 2022-12-05. Retrieved 2022-09-27.