OpenEnv documentation
Your First Environment
Get Started
Guides
Tutorials
OverviewHello WorldTrain a reasoning model with TRLMCP EnvironmentsRubricsRL Training with TRLRL Training with UnslothEvaluating with Inspect AIRL Training with an Agentic HarnessCoding Agent Training with TRLSFT Training with EnvironmentsTraining a Real Coding Agent (Pi)
Environments
API Reference
Project
Your First Environment
This page is a condensed preview. For the end-to-end walk-through — including Docker packaging,
openenv.yaml, and Hugging Face Space deployment — see the full environment builder guide.
Overview
Building an OpenEnv environment involves:
- Define your models -
Action,Observation, andStatetypes - Implement the environment - Core logic in a Python class
- Create the server - FastAPI wrapper for HTTP access
- Package for deployment - Docker container and manifest
Quick Example
Here’s a minimal environment that echoes back messages. Reward and done are fields on the Observation — step returns an observation, not a tuple.
from openenv.core.env_server.interfaces import Environment
from openenv.core.env_server.types import Action, Observation, State
class EchoAction(Action):
message: str
class EchoObservation(Observation):
echo: str
class EchoState(State):
last_message: str = ""
class EchoEnvironment(Environment[EchoAction, EchoObservation, EchoState]):
def reset(self, seed=None, episode_id=None, **kwargs) -> EchoObservation:
self._state = EchoState()
return EchoObservation(echo="Ready!")
def step(self, action: EchoAction, timeout_s=None, **kwargs) -> EchoObservation:
self._state.last_message = action.message
return EchoObservation(echo=action.message, reward=0.0, done=False)
@property
def state(self) -> EchoState:
return self._stateNext Steps
- Concepts - Deep dive into environment structure
- Packaging & Deploying - Deploy to Docker and HF Spaces