Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks. - mlabonne/llm-course

Supercharge Your Dev Stack: 200+ AI Models, One API What is AI/ML API? AI/ML API is a game-changing platform for developers and SaaS entrepreneurs looking to integrate cutting-edge AI capabilities into their products. It offers a single point of access to over 200 state-of-the-art AI models, covering everything from NLP to computer vision. Key Features for Developers: Extensive Model Library: 200+ pre-trained models for rapid prototyping and deployment Customization Options: Fine-tune models to fit your specific use case Developer-Friendly Integration: RESTful APIs and SDKs for seamless incorporation into your stack Serverless Architecture: Focus on coding, not infrastructure management Advantages for SaaS Entrepreneurs: Rapid Time-to-Market: Leverage advanced AI without building from scratch Scalability: From MVP to enterprise-grade solutions, AI/ML API grows with your business Cost-Efficiency: Pay-as-you-go pricing model reduces upfront investment Competitive Edge: Stay ahead with continuously updated AI models

Document about LM Studio

Optimum is an extension of Transformers that provides a set of performance optimization tools to train and run models on targeted hardware with maximum efficiency. It is also the repository of small, mini, tiny models.

BERT model home on github

Excellent document about BERT transformers / models and their parameters: - L=number of layers. - H=size of the hidden layer = number of vectors for each word in the sentence. - A = Number of self-attention heads - Total parameters.

Repository of all Bert models, including small. Start using this model for testing.

Comparison of efficiency of all LLM models on hugging face

Various methods to run LLM models locally hugging face is only one of them.

GGML quantized models. They would let you leverage CPU and system RAM, instead of having to rely on a GPU’s. This could save you a fortune, especially if go for some used AMD Epyc platforms. This could be more viable for the larger models, especially the 30B/65B parameters models which would still press or exceed the VRAM on the P40.

High level how to train a model

Research community developing various code models, small and big. Models may not be instruct

Another open source small (1B) model.

They have the 1.3B version!!! This may be the best to start with Newspeak. Should work train even on huggingcface

Another possible model. For coding capabilities, Deepseek Coder achieves state-of-the-art performance among open-source code models on multiple programming languages and various benchmarks.

With the optimizers of bitsandbytes (like 8 bit AdamW), you would need 2 bytes per parameter, or 14 GB of GPU memory.

Another potential model to use for Newspeak, but it is NOT open source. Adventage: 2.5B params, so should be usable in small GPUs

training a model like Llama with 2.7 billion parameters outperformed a larger model like Vicuna with 13 billion parameters. Especially when considering resource consumption, this might be a good alternative to using a 7B Foundation model instead of a full-blown ChatGPT. The best price-to-performance base model for our use case turned out to be Mistral 7b. The model is compact enough to fit into an affordable GPU with 24GB VRAM and outperforms the other models with 7B parameters.

Comparison of LLM models for coding

Open source with lots of information. Uses Multiple undrelying models. Not sure how I would train for it

The Mixtral model is new, and seems to be good. Click on “Demo“ to test it

Article has comparison with other code-LLM models

Includes code generation models

List of LLM models on Wikipedia

Use the Bert model to train on Yelp dataset

Hi level only talk about training for a language

Describes how to train a new language (desperanto) model.