Artificial Intelligence
(4)
๐๐ซ๐๐ง๐ฌ๐๐จ๐ซ๐ฆ๐๐ซ ๐๐ฑ๐ฉ๐ฅ๐๐ข๐ง๐๐ซ
Beautiful visualization of the inner workings of a Transformer.
An interactive visualization tool showing you how transformer models work in large language models (LLM) like GPT.
โ๐๐น๐ด๐ผ๐ฟ๐ถ๐๐ต๐บ๐ ๐ณ๐ผ๐ฟ ๐๐ฒ๐ฐ๐ถ๐๐ถ๐ผ๐ป ๐ ๐ฎ๐ธ๐ถ๐ป๐ดโ, MIT publishing book is freely available.
Book: https://algorithmsbook.com/
๐ฅ๐ง๐ต๐ฒ ๐๐ ๐ฆ๐ฐ๐ถ๐ฒ๐ป๐๐ถ๐๐
Paper: https://arxiv.org/abs/2408.06292
GitHub: https://github.com/SakanaAI/AI-Scientist
๐๐ผ๐ฑ๐ถ๐ป๐ด ๐ฎ ๐ ๐๐น๐๐ถ๐บ๐ผ๐ฑ๐ฎ๐น (๐ฉ๐ถ๐๐ถ๐ผ๐ป) ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐ ๐ผ๐ฑ๐ฒ๐น ๐ณ๐ฟ๐ผ๐บ ๐๐ฐ๐ฟ๐ฎ๐๐ฐ๐ต ๐ถ๐ป ๐ฃ๐๐ง๐ผ๐ฟ๐ฐ๐ต.
Topics:
Transformer model (Embeddings, Positional Encoding, Multi-Head Attention, Feed Forward Layer, Logits, Softmax)
Vision Transformer model
Contrastive learning (CLIP, SigLip)
Numerical stability of the Softmax and the Cross Entropy Loss
Rotary Positional Embedding
Multi-Head Attention
Grouped Query Attention
Normalization layers (Batch, Layer and RMS)
KV-Cache (prefilling and token generation)
Attention masks (causal and non-causal)
Weight tying
Top-P Sampling and Temperature
and much more!
https://youtu.be/vAmKB7iPkWw?si=Lp4xV3xJlMgQfdtL
Full coding of a Multimodal (Vision) Language Model from scratch using only Python and PyTorch. We will be coding the PaliGemma Vision Language Model from sc...
๐ฅIntroducing Meta Segment Anything Model 2 (SAM 2) โ the first unified model for real-time, promptable object segmentation in images & videos.
SAM 2 is available today under Apache 2.0 so that anyone can use it to build their own experiences
Details โก๏ธ
https://ai.meta.com/blog/segment-anything-2
Blog Post: https://ai.meta.com/blog/segment-anything-2/Paper: https://ai.meta.com/research/publications/sam-2-segment-anything-in-images-and-videos/Demo: htt...
๐ฅhttps://artificialanalysis.ai/ is a great site that benchmarks quality/speed/price of different LLM API providers to help developers pick which models to use.
๐ฅAn excellent new book to learn the concepts of Deep Learning.
Topics include fundamental building blocks, Transformers, GNNs, RL, diffusion models, and more.
FREE PDF: https://udlbook.github.io/udlbook/
GraphRAG, a graph-based approach to retrieval-augmented generation (RAG) that significantly improves question-answering over private or previously unseen datasets, is now available on GitHub.
https://www.microsoft.com/en-us/research/blog/graphrag-new-tool-for-complex-data-discovery-now-on-github/
GraphRAG, a graph-based approach to retrieval-augmented generation (RAG) that significantly improves question-answering over private or previously unseen datasets, is now available on GitHub. Learn more:
๐ฅ๐๐๐ ๐๐๐๐ ๐๐๐๐ ๐ฉ๐๐ฉ๐๐ซ๐ฌ
This repository is a curated collection of the most exciting and influential CVPR 2024 papers!
https://github.com/SkalskiP/top-cvpr-2024-papers
This repository is a curated collection of the most exciting and influential CVPR 2024 papers. ๐ฅ [Paper + Code + Demo] - SkalskiP/top-cvpr-2024-papers
Together AI and Morph Labs collaborated to create an excellent blog post on tuning models for RAG (Retrieval Augmented Generation). RAG fine-tuning combines code retrieval with model training, addressing the limitations of outdated knowledge and hallucinations in LLMs.
Large Language Models (LLMs) have shown promising capabilities on multiple applications such as code generation, task planning, and document understanding. Despite the impressive performance, these models often fall short due to two main reasons: hallucinations and outdated knowledge in the models.....
๐ฅ๐๐๐ญโ๐ฌ ๐ซ๐๐ฉ๐ซ๐จ๐๐ฎ๐๐ ๐๐๐-๐ (๐๐๐๐) by Andrej Karpathy (former OpenAI scientist and Tesla's former head of AI)
๐ฝ๏ธ New 4 hour video lecture on YouTube:
https://youtu.be/l8pRSuU81PU?si=M1kznmR5XSiYW-Qz
We reproduce the GPT-2 (124M) from scratch. This video covers the whole process: First we build the GPT-2 network, then we optimize its training to be really...
๐ฅ๐๐ฐ๐๐ง๐ is the newest Alibaba's open source large language model. It slightly surpasses Llama 3 70B on benchmark performance in English while being a better multilingual model.
Blog:
https://qwenlm.github.io/blog/qwen2/
๐๐ก๐ ๐๐ข๐ฌ๐ ๐จ๐ ๐๐ ๐๐ง๐ญ๐ข๐ ๐๐๐ญ๐ซ๐ข๐๐ฏ๐๐ฅ-๐๐ฎ๐ ๐ฆ๐๐ง๐ญ๐๐ ๐๐๐ง๐๐ซ๐๐ญ๐ข๐จ๐ง (๐๐๐) ๐ข๐ง ๐๐ซ๐ญ๐ข๐๐ข๐๐ข๐๐ฅ ๐๐ง๐ญ๐๐ฅ๐ฅ๐ข๐ ๐๐ง๐๐ ๐๐
In the rapidly developing fields of data science and Artificial Intelligence (AI), the search for increasingly effective systems is also increasing significantly. The development of Agentic Retrieval-Augmented Generation (RAG) is among the most revolutionary developments of recent times. This strategy is set to completely transform the way information is used and managed, offering a substantial improvement over current RAG systems.
https://www.marktechpost.com/2024/05/28/the-rise-of-agentic-retrieval-augmented-generation-rag-in-artificial-intelligence-ai/
The Rise of Agentic Retrieval-Augmented Generation (RAG) in Artificial Intelligence AI
๐๐ซ๐๐ฉ๐ก๐๐๐ (Graph-based Retrieval Augmented Generation) enhances the traditional Retrieval Augmented Generation (RAG) method by integrating knowledge graphs (KGs) or graph databases with large language models (LLMs). It leverages the structured nature of graph databases to organize data as nodes and relationships, enabling more efficient and accurate retrieval of relevant information to provide better context to LLMs for generating responses.
https://gradientflow.substack.com/p/graphrag-design-patterns-challenges
Subscribe โข Previous Issues Enhancing RAG with Knowledge Graphs: Blueprints, Hurdles, and Guidelines By Ben Lorica and Prashanth Rao. GraphRAG (Graph-based Retrieval Augmented Generation) enhances the traditional Retrieval Augmented Generation (RAG) method by integrating knowledge graphs (
๐๐๐ ๐๐๐ฌ๐๐๐ซ๐๐ก๐๐ซ
-GPT based autonomous agent that does online comprehensive research on any given topic.
-GPT Researcher supports all major LLM providers.
Repo:
https://github.com/assafelovic/gpt-researcher
๐๐ข๐ฌ๐ญ๐ซ๐๐ฅ ๐
๐ข๐ง๐๐ญ๐ฎ๐ง๐
Mistral released an official repository to fine-tune its models.
GitHub Repo:
https://github.com/mistralai/mistral-finetune
๐๐ข๐๐๐ฎ๐ฌ๐ข๐จ๐ง ๐๐จ๐๐๐ฅ๐ฌ
Notes on the theory behind models like Stable Diffusion and their applications.
https://andrewkchan.dev/posts/diffusion.html
ContentsDiffusion ModelsNotes on the theory behind models like Stable Diffusion and their applications. I spent 2022 learning to draw and was blindsided by the rise of AI art models like Stable Diffusion. Suddenly, the computer was a better artist than I could ever hope to be. It's been two years, a...
๐๐ฅ๐ข๐
A snappy, keyboard-centric terminal user interface for interacting with large language models. Chat with ChatGPT, Claude, Llama 3, Phi 3, Mistral, Gemma and more.
Repo:
https://github.com/darrenburns/elia
๐ฅ๐ฅ๐ฅ๐๐ฆ๐๐ ๐ข๐ฆ๐ฉ๐ฅ๐๐ฆ๐๐ง๐ญ๐๐ ๐๐ซ๐จ๐ฆ ๐ฌ๐๐ซ๐๐ญ๐๐ก
This great tutorial shows every step of reconstructing Llama 3 and running the trained weights.
Repo:
https://github.com/naklecha/llama3-from-scratch
๐๐จ๐ฅ๐ฆ๐จ๐ ๐จ๐ซ๐จ๐ฏ ๐๐ซ๐ง๐จ๐ฅ๐ ๐๐๐ญ๐ฐ๐จ๐ซ๐ค๐ฌ (๐๐๐) ๐๐๐ฉ๐๐ซ ๐๐ฑ๐ฉ๐ฅ๐๐ข๐ง๐๐ - An exciting new paradigm for Deep Learning?
https://youtu.be/7zpz_AlFW2w
This is a paper breakdown video of the paper: Kolmogorov Arnold Networks, which brilliantly provides an alternative to standard Multi Layer Perceptrons. The ...
๐ฅ๐๐๐ฏ๐๐ง๐๐๐ ๐๐๐ ๐๐ซ๐จ๐ฆ ๐๐๐ซ๐ง๐๐ ๐ข๐ ๐๐๐ฅ๐ฅ๐จ๐ง ๐๐ง๐ข๐ฏ๐๐ซ๐ฌ๐ข๐ญ๐ฒ! (2024)
Great recent NLP topics like prompting, fine-tuning and instruction-tuning, retrieval and RAG, ensembling and mixture of experts (MoE), and more.
One of the best NLP courses on the web:
https://www.youtube.com/playlist?list=PL8PYTP1V4I8D0UkqW2fEhgLrnlDW9QK7z
This year, more and more developers are talking about AI agents - autonomous or semi-autonomous systems capable of handling a wider range of tasks and making decisions on their own. Unlike co-pilots, agents have a higher degree of autonomy and can take proactive actions based on their goals and understanding of the environment. They can complete tasks without constant human intervention, learning and adapting based on their interactions and experiences.
https://gradientflow.substack.com/p/agentic-ai-challenges-and-opportunities
Subscribe โข Previous Issues Navigating the Complex World of AI Agents Last year, the buzz in the AI community revolved around the concept of AI co-pilots - systems designed to work alongside humans, assisting them in tasks and decision-making processes. These co-pilots, such as GitHub Copilot for ...
๐ฅ๐ฅ๐ฅ๐๐๐ญ๐ ๐๐๐ข๐๐ง๐๐ ๐๐ ๐๐ง๐ญ, a Google labs experiment! An experiment to build an AI generated Colab notebook that handles data cleaning, data exploration, plotting, Q&A on data, and predictive modeling.
โ Helps with complex tasks like planning, and error correction.
โ Helps with data science tasks like predictive modeling.
โ Outputs an AI-generated Colab notebook based on your prompt.
Try it at https://labs.google.com/code/
๐ฅ๐๐ง๐๐ซ๐๐๐ข๐๐ฅ๐ ๐ฌ๐๐ซ๐ข๐๐ฌ ๐จ๐ ๐๐ ๐ฅ๐๐๐ญ๐ฎ๐ซ๐๐ฌ ๐๐ซ๐จ๐ฆ ๐๐ ๐๐ฎ๐ฌ๐ญ๐ข๐ง ๐จ๐ง ๐๐๐ ๐๐ง๐ ๐๐๐๐ฌ
It gives descent synopses of modern NLP topics and recent ones like RLHF, instruction-tuning, few-shot prompting, chain-of-thought, and more.
Lectures:
https://www.youtube.com/playlist?list=PLofp2YXfp7TZZ5c7HEChs0_wfEfewLDs7
๐ฅ๐๐ซ๐๐ง๐ข๐ญ๐ ๐๐จ๐๐ ๐๐จ๐๐๐ฅ๐ฌ: ๐ ๐
๐๐ฆ๐ข๐ฅ๐ฒ ๐จ๐ ๐๐ฉ๐๐ง ๐
๐จ๐ฎ๐ง๐๐๐ญ๐ข๐จ๐ง ๐๐จ๐๐๐ฅ๐ฌ ๐๐จ๐ซ ๐๐จ๐๐ ๐๐ง๐ญ๐๐ฅ๐ฅ๐ข๐ ๐๐ง๐๐
- IBM has released four variations of the Granite code model.
- Ranging in size from 3 to 34B parameters
- Trained on 3 to 4T tokens sourced from ๐๐๐ ๐ฉ๐ซ๐จ๐ ๐ซ๐๐ฆ๐ฆ๐ข๐ง๐ ๐ฅ๐๐ง๐ ๐ฎ๐๐ ๐๐ฌ
- The models have outperformed other comparable models like Code Llama and Llama 3 in many tasks.
- Repo:
https://github.com/ibm-granite/granite-code-models
-Paper:
https://arxiv.org/abs/2405.04324
๐ฅ๐๐จ๐ฅ๐ฆ๐จ๐ ๐จ๐ซ๐จ๐ฏ-๐๐ซ๐ง๐จ๐ฅ๐ ๐๐๐ญ๐ฐ๐จ๐ซ๐ค ๐๐จ๐ซ ๐๐๐ข๐ง๐๐จ๐ซ๐๐๐ฆ๐๐ง๐ญ ๐๐๐๐ง๐ข๐ง๐ , ๐ข๐ง๐ข๐ญ๐ข๐๐ฅ ๐๐ฑ๐ฉ๐๐ซ๐ข๐ฆ๐๐ง๐ญ๐ฌ
This small project test the novel architecture Kolmogorov-Arnold Networks (KAN) in the reinforcement learning paradigm to the CartPole problem.
Repo:
https://github.com/riiswa/kanrl
๐ฅ๐ฑ๐๐๐๐: ๐๐ฑ๐ญ๐๐ง๐๐๐ ๐๐จ๐ง๐ ๐๐ก๐จ๐ซ๐ญ-๐๐๐ซ๐ฆ ๐๐๐ฆ๐จ๐ซ๐ฒ
Sepp Hochreiter, who invented the LSTM, just dropped a new LLM architecture!
-The xLSTM architecture is shown to be efficient at handling different aspects of long context problems.
-Major component is a new parallelizable LSTM.
-One of the major weaknesses of prior LSTMs was the sequential nature (can't be done at once)
Check the paper for more interesting insights and results:
https://arxiv.org/abs/2405.04517
๐ฅ๐ฅ๐ฅ๐๐ฐ๐๐ฌ๐จ๐ฆ๐ ๐๐๐(๐๐จ๐ฅ๐ฆ๐จ๐ ๐จ๐ซ๐จ๐ฏ-๐๐ซ๐ง๐จ๐ฅ๐ ๐๐๐ญ๐ฐ๐จ๐ซ๐ค)
A curated list of awesome libraries, tutorials, papers, and other resources related to Kolmogorov-Arnold Network (KAN). This repository aims to be a comprehensive and organized collection that will help researchers and developers in the world of KAN!
Repo: https://github.com/mintisan/awesome-kan
๐ฅ๐ฅ๐ฅ๐๐๐-๐๐๐
The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling
Repo:
https://github.com/AdityaNG/kan-gpt
๐๐๐๐ ๐๐๐๐ ๐๐ฎ๐ญ๐ฌ๐ญ๐๐ง๐๐ข๐ง๐ ๐๐๐ฉ๐๐ซ ๐๐ฐ๐๐ซ๐๐ฌ (International Conference on Learning Representations)
1. Generalization in diffusion models arises from geometry-adaptive harmonic representations
2. Learning Interactive Real-World Simulators
3. Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors
4. Protein Discovery with Discrete Walk-Jump Sampling
5. Vision Transformers Need Registers
Blog:
https://blog.iclr.cc/2024/05/06/iclr-2024-outstanding-paper-awards/
May 6 2024 ICLR 2024 Outstanding Paper Awards Yisong Yue ICLR 2024 Awards Committee: Eunsol Choi, Katja Hofmann, Ming-Yu Liu, Nan Jiang, Stephan Gรผnnemann, Suvrit Sra, Thomas Kipf, Volkan Cevher (This post is written by the Awards Committee, lightly edited by the Program Chairs.) Selection Process ...
Be the first to know and let us send you an email when sistemas.ai posts news and promotions. Your email address will not be used for any other purpose, and you can unsubscribe at any time.
Send a message to sistemas.ai:
๐ฅAnimated Line Chart with matplotlib! Tutorial: https://python-graph-gallery.com/web-animated-line-chart-with-annotation/ #SistemasAi #datascience
๐ฅ๐๐ป๐ฐ๐ถ๐ฒ๐ป๐ ๐๐ต๐ถ๐ป๐ฒ๐๐ฒ ๐๐ฒ๐ฟ๐ฟ๐ฎ๐ฐ๐ผ๐๐๐ฎ ๐๐ฎ๐ฟ๐ฟ๐ถ๐ผ๐ฟ ๐ฏ๐ฟ๐ผ๐๐ด๐ต๐ ๐๐ผ ๐น๐ถ๐ณ๐ฒ ๐ฏ๐ ๐ฐ๐ผ๐ฑ๐ฒ๐ฟ ๐๐๐ถ๐ป๐ด ๐ฎ๐ฟ๐๐ถ๐ณ๐ถ๐ฐ๐ถ๐ฎ๐น ๐ถ๐ป๐๐ฒ๐น๐น๐ถ๐ด๐ฒ๐ป๐ฐ๐ฒ ๐ฎ๐น๐ด๐ผ๐ฟ๐ถ๐๐ต๐บ๐ Guerreros 'Terracota' de la antigua China traรญdos a la vida usando inteligencia artificial. Artรญculo: https://www.scmp.com/abacus/tech/article/3099065/ancient-chinese-terracotta-warrior-brought-life-coder-using-artificial #SistemasAi #Ai #artificialintelligence #inteligenciaartificial #ComputerVision
๐ง๐๐ผ ๐ง๐ต๐ผ๐๐๐ฎ๐ป๐ฑ ๐ฌ๐ฒ๐ฎ๐ฟ๐ ๐ผ๐ณ ๐๐น๐ผ๐ฏ๐ฎ๐น ๐ง๐ฒ๐บ๐ฝ๐ฒ๐ฟ๐ฎ๐๐๐ฟ๐ฒ๐ ๐ถ๐ป ๐ง๐๐ฒ๐ป๐๐ ๐ฆ๐ฒ๐ฐ๐ผ๐ป๐ฑ๐ ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ๐ฅ Visualizaciรณn por bgregory98 vรญa Reddit. #SistemasAi #Data
๐ก๐ฒ๐ ๐ฝ๐ฎ๐ฝ๐ฒ๐ฟ: ๐๐ผ๐ป๐๐ผ๐น๐๐๐ถ๐ผ๐ป๐ฎ๐น ๐ข๐ฐ๐ฐ๐๐ฝ๐ฎ๐ป๐ฐ๐ ๐ก๐ฒ๐๐๐ผ๐ฟ๐ธ๐ Nuevo 'paper' destacado en el #ECCV2020 (23-28 de agosto). รltimamente, "๐ถ๐บ๐ฝ๐น๐ถ๐ฐ๐ถ๐ ๐ป๐ฒ๐๐ฟ๐ฎ๐น ๐ฟ๐ฒ๐ฝ๐ฟ๐ฒ๐๐ฒ๐ป๐๐ฎ๐๐ถ๐ผ๐ป๐" tienen un alto grado de aceptaciรณn en sistemas de reconstrucciones 3D basados en modelos de aprendizaje automรกtico. Comparado con otros papers, ๐๐ผ๐ป๐๐ผ๐น๐๐๐ถ๐ผ๐ป๐ฎ๐น ๐ข๐ฐ๐ฐ๐๐ฝ๐ฎ๐ป๐ฐ๐ ๐ก๐ฒ๐๐๐ผ๐ฟ๐ธ๐ se basa en una "implicit neural representation" aรบn mรกs flexible para una mejor reconstrucciรณn al detalle de objetos y escenas 3D a gran escala que combina "convolutional encoders" con "implicit occupancy decoders". Si estรกs interesado en reconstrucciones 3D a gran escala... ๐๐ผ๐ป๐๐ผ๐น๐๐๐ถ๐ผ๐ป๐ฎ๐น ๐ข๐ฐ๐ฐ๐๐ฝ๐ฎ๐ป๐ฐ๐ ๐ก๐ฒ๐๐๐ผ๐ฟ๐ธ๐ es lo mejor que hay hasta el momento. Cรณdigo y data incluรญdos! Abstract: https://arxiv.org/abs/2003.04618 Paper: https://arxiv.org/pdf/2003.04618 Project page: https://pengsongyou.github.io/conv_onet Cรณdigo y data: https://github.com/autonomousvision/convolutional_occupancy_networks YouTube: https://youtu.be/EmauovgrDSM #SistemasAi #Ai #inteligenciaartificial #DeepLearning #artificialintelligence #MachineLearning #ComputerVision
Imรกgenes originales utilizadas para animar el primer videojuego de Mortal Kombat ๐ Hoy en dรญa usando Deep Learning se pueden producir personajes de video-juegos que se asemejan a la realidad. #SistemasAi #Animation #DeepLearning #ComputerVision
Want your business to be the top-listed Media Company?
