Skip to content

A Collection of Competitive Text-Based Games for Language Model Evaluation and Reinforcement Learning

License

Notifications You must be signed in to change notification settings

josecodes/TextArena

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PyPI version Discord Website

TextArena  

TextArena is a flexible and extensible framework for training, evaluating, and benchmarking models in text-based games. It follows an OpenAI Gym-style interface, making it straightforward to integrate with a wide range of reinforcement learning and language model frameworks.


Example

Installation

Install TextArena directly from PyPI:

pip install textarena

Install enchant on ubuntu:

apt install enchant2

Play Offline

import textarena as ta

# Initialize agents
agents = {
    0: ta.agents.OpenRouterAgent(model_name="GPT-4o-mini"),
    1: ta.agents.OpenRouterAgent(model_name="anthropic/claude-3.5-haiku"),
}

# Initialize environment from subset and wrap it
env = ta.make(env_id="BalancedSubset-v0")
env = ta.wrappers.LLMObservationWrapper(env=env)
env = ta.wrappers.SimpleRenderWrapper(
    env=env,
    player_names={0: "GPT-4o-mini", 1: "claude-3.5-haiku"},
)

env.reset()
done = False
while not done:
    player_id, observation = env.get_observation()
    action = agents[player_id](observation)
    done, info = env.step(action=action)
rewards = env.close()

Implementation Status

Single-Player Games

Game Name Offline Play Online Play Documentation
CarPuzzle
Chess
ConnectFour
Crosswords link
FifteenPuzzle link
GuessTheNumber link
GuessWho link
Hangman link
LogicPuzzle link
MathProof
Minesweeper link
Sudoku link
TowerOfHanoi link
TwentyQuestions link
WordLadder link
WordSearch link

Two-Player Games

Game Name Offline Play Online Play Documentation
Battleship link
Brass
CarPuzzle
Chess link
ConnectFour link
Debate link
DontSayIt link
IteratedPrisonersDilemma link
Jaipur
LetterAuction
LiarsDice link
Mastermind link
MathProof
MemoryGame
Negotiation link
Poker link
ScenarioPlanning
SpellingBee link
SpiteAndMalice link
Stratego link
Taboo
Tak link
UltimateTicTacToe link
TruthAndDeception link
WordChains link

Multi-Player Games

Game Name Offline Play Players Online Play Documentation
Diplomacy 3+
7 Wonders 3+
Bohnanza 3+
Codenames 4+
Negotiation 3+
Poker 3+
Risk 3+
SettlersOfCatan 3-4
TerraformingMars 1-5
Werewolf 5+

About

A Collection of Competitive Text-Based Games for Language Model Evaluation and Reinforcement Learning

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 98.0%
  • Other 2.0%