Skip to content

Latest commit

 

History

History
96 lines (77 loc) · 8.7 KB

README.md

File metadata and controls

96 lines (77 loc) · 8.7 KB

Basic concepts of LLMs

Architecture

Useful links

Papers

  • Language Models are Unsupervised Multitask Learners. [Paper][Code]
  • Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning. [Paper][Code]
  • NIPS2017 Attention Is All You Need. [Paper][Code]
  • 11 Oct 2018 arxiv BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. [Paper][Code]
  • ICML22 What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? [Paper][Code]

Tokenizer

Useful links

Papers

  • ACL2016 Neural Machine Translation of Rare Words with Subword Units. [Paper]
  • 26 Sep 2016 arxiv Google’s neural machine translation system: Bridging the gap between human and machine translation. [Paper]
  • ACL2018 Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates. [Paper]

Others

LLM4EDA

Generation view

Papers

  1. Code generation

    1. Codex
    2. AlpaCode
    3. (MLCAD 2023) Chateda: A large language model powered autonomous agent for eda. [Paper]
    4. (23 May 2023 arxiv) Chipgpt: How far are we from natural language hardware design. [Paper]
    5. (ICCAD23 invited) Verilogeval: Evaluating large language models for verilog code generation. [Paper][Code]
    6. (ICCAD23) Gpt4aigchip: Towards next-generation ai accelerator design automation via large language models. [Paper]
    7. (MLCAD23) Chip-chat: Challenges and opportunities in conversational hardware design. [Paper]
    8. (8 Nov 2023 arxiv) Autochip: Automating hdl generation using llm feedback. [Paper][Code]
    9. (ASP-DAC24) Rtllm: An open-source benchmark for design rtl generation with large language model. [Paper][Code]
    10. (DATE23) Benchmarking large language models for automated verilog rtl code generation. [Paper][Code]

  2. Code Verification & Analysis

    1. (31 Oct 2023 arxiv) Chipnemo: Domain-adapted llms for chip design. [Paper]
    2. (28 Nov 2023 arxiv) Rtlfixer: Automatically fixing rtl syntax errors with large language models. [Paper]
    3. (DAC21) Autosva: Democratizing formal verification of rtl module interactions. [Paper]
    4. (24 Jun 2023 arxiv) Llm-assisted generation of hardware assertions. [Paper]
    5. (21 Aug 2023 arxiv) Unlocking hardware security assurance: The potential of llms. [Paper]
    6. (14 Aug 2023 arxiv) Divas: An llm-based end-to-end framework for soc security analysis and policy-based protection. [Paper]
    7. (2 Feb 2023 arxiv) Fixing hardware security bugs with large language models. [Paper]

  3. Specification Generation

    1. (24 Jan 2024 arxiv) SpecLLM: Exploring Generation and Review of VLSI Design Specification with Large Language Model. [Paper][Code]

Optimization view

Useful links

Papers

  • (NIPS2011) Algorithms for Hyper-Parameter Optimization. [Paper]
  • (ACL2023) Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers. [Paper][Code]
  1. Combinatorial / Discrete Problem
    1. (ICLR24) Prompt Type1 LARGE LANGUAGE MODELS AS OPTIMIZERS. [Paper][Code]
    2. (ICLR24) Bayesian Optimization Type3 LARGE LANGUAGE MODELS TO ENHANCE BAYESIAN OPTIMIZATION. [Paper]Code]
    3. (19 Jan 2024 arxiv) Evolutionary algorithm Type3 A match made in consistency heaven: when large language models meet evolutionary algorithms. [Paper]
    4. (29 Oct arxiv) Evolutionary algorithm Type3 Large Language Models as Evolutionary Optimizers. [Paper]
    5. (8 Oct arxiv) Prompt Type1 Towards Optimizing with Large Language Model. [Paper]
  2. Numerical / Continuous Problem
    1. (ICLR24) Prompt Type1 LARGE LANGUAGE MODELS AS OPTIMIZERS. [Paper][Code]
    2. (Nature) Prompt Type1 Mathematical discoveries from program search with large language models. [Paper]
    3. (8 Jul arxiv) Prompt Type3 Large Language Models for Supply Chain Optimization. [Paper][Code]
    4. (NIPS23) Prompt Type1 Using Large Language Models for Hyperparameter Optimization. [Paper]
    5. (19 Jan 2024 arxiv) Evolutionary algorithm Type3 A match made in consistency heaven: when large language models meet evolutionary algorithms. [Paper]
    6. (22 Nov 2023 arxiv) Reinforcement learning Type3 Large Language Model is a Good Policy Teacher for Training Reinforcement Learning Agents. [Paper][Code]
    7. (25 May 2023 arxiv) Reinforcement learning Type1 Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory. [Paper][Code]
    8. (29 Oct arxiv) Evolutionary algorithm Type3 Large Language Models as Evolutionary Optimizers. [Paper]

Acknowledgment

  1. Zhao W X, Zhou K, Li J, et al. A survey of large language models[J]. arXiv preprint arXiv:2303.18223, 2023. [Paper][Code]
  2. Zhong R, Du X, Kai S, et al. LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation[J]. arXiv preprint arXiv:2401.12224, 2023. [Paper]Code]