# ModelEndpoint Source: /reference/modelendpoints/ A ModelEndpoint is somewhere a request can be served: one replica of a ModelDeployment, or a model at a provider like Together or Groq. It describes a backend well enough for a gateway to talk to it without knowing where it came from, so a ModelService can fan over endpoints Modelplane runs and endpoints from a provider. Modelplane composes one per replica. You write them by hand for anything it doesn't run. Apply instances as `apiVersion: modelplane.ai/v1alpha1`, `kind: ModelEndpoint`. [Concept guide: Route to External Providers](/models/model-endpoint/index.md) ## Example ```yaml # ModelDeployment composes a ModelEndpoint per replica. Write one by hand only # to register a model Modelplane doesn't run, like this one at Together. apiVersion: modelplane.ai/v1alpha1 kind: ModelEndpoint metadata: name: together-qwen-72b namespace: ml-team labels: modelplane.ai/endpoint: together-qwen-72b spec: # Scheme and host, no path. An https origin gets TLS originated to it. The # host must be a name; an address stops the gateway applying the model # rewrite, the credential and priority failover. origin: https://api.together.xyz api: # OpenAI (the default) or Anthropic. The gateway translates an Anthropic # caller's request for an OpenAI backend, but an Anthropic backend serves # only Anthropic callers. schema: OpenAI # The path this backend serves that API under. /v1 for most, /openai/v1 for # Groq, a per-replica path for a Modelplane-composed endpoint. prefix: /v1 # The name this backend knows the model by. Unset, the caller's model name # passes through unchanged. model: Qwen/Qwen2.5-72B-Instruct-Turbo # This backend's credential, attached by the gateway on the way out. APIKey # sends the key in the Secret as a bearer token, or in x-api-key to an # Anthropic backend. credential: method: APIKey apiKey: secretRef: name: together-api-key ``` ## Definition The CompositeResourceDefinition this reference is generated from, with the complete OpenAPI schema, validation rules, and defaults: ```yaml apiVersion: apiextensions.crossplane.io/v2 kind: CompositeResourceDefinition metadata: name: modelendpoints.modelplane.ai spec: group: modelplane.ai names: categories: [crossplane, modelplane, models] kind: ModelEndpoint plural: modelendpoints shortNames: [me] scope: Namespaced versions: - name: v1alpha1 served: true referenceable: true additionalPrinterColumns: - name: ORIGIN type: string jsonPath: .spec.origin - name: MODEL type: string jsonPath: .spec.model schema: openAPIV3Schema: description: >- A ModelEndpoint is somewhere a request can be served: one replica of a ModelDeployment, or a model at a provider like Together or Groq. It describes a backend well enough for a gateway to talk to it without knowing where it came from, so a ModelService can fan over endpoints Modelplane runs and endpoints from a provider. Modelplane composes one per replica. You write them by hand for anything it doesn't run. type: object required: [spec] properties: spec: type: object required: [origin] properties: origin: type: string description: >- Scheme and host of the backend, with no path: an https origin gets TLS originated to it. A port is only needed for a non-default one. The host must be a name, never an address. minLength: 1 maxLength: 2048 x-kubernetes-validations: - rule: "self.startsWith('http://') || self.startsWith('https://')" message: spec.origin must start with http:// or https://. # Every rule on a field is evaluated, so this one has to tolerate # an origin the rule above already rejected rather than indexing # past the end of the split and reporting a CEL runtime error. # It also catches a trailing slash, which would otherwise join # with api.prefix to make a double slash. - rule: "!self.contains('://') || self.split('://')[1].split('/').size() == 1" message: spec.origin must be scheme and host only; put the API's path in spec.api.prefix. # A scheme with no host, like https://, otherwise slips through: # the rules above are satisfied by the scheme alone. - rule: "!self.contains('://') || self.split('://')[1] != ''" message: spec.origin must include a host. # A port that isn't one number would reach the function and crash # int(port) while composing the backend, taking the whole route # down. An IPv6 literal is left to the address rule below. - rule: "!self.contains('://') || self.split('://')[1].startsWith('[') || !self.split('://')[1].contains(':') || (self.split('://')[1].split(':').size() == 2 && self.split('://')[1].split(':')[1].matches('^[0-9]{1,5}$') && int(self.split('://')[1].split(':')[1]) >= 1 && int(self.split('://')[1].split(':')[1]) <= 65535)" message: spec.origin's port must be a number from 1 to 65535. # An IP literal is the one host Envoy AI Gateway routes to but # silently won't rewrite for, so reject it here rather than let a # caller's model name and no credential reach a backend. A # bracket is an IPv6 literal; the regex is a dotted-quad IPv4, # after dropping any :port. - rule: "!self.contains('://') || (!self.split('://')[1].startsWith('[') && !self.split('://')[1].split(':')[0].matches('^[0-9]+([.][0-9]+){3}$'))" message: spec.origin host must be a name, not an IP address. api: type: object description: >- The API this backend speaks, and where it serves it. Defaults to the OpenAI API under /v1, which most providers serve. # So omitting api entirely still yields the OpenAI/v1 defaults, # not an empty object; the field defaults below only fill a # present api. default: {} properties: schema: type: string description: >- The API the backend speaks. A gateway translates an Anthropic request for an OpenAI backend, but not the reverse: an Anthropic backend serves only Anthropic callers, and an OpenAI request routed to it fails. default: OpenAI enum: [OpenAI, Anthropic] prefix: type: string description: >- The path the backend serves that API under: /v1 for most, /openai/v1 for Groq, and a per-replica path for a Modelplane-composed endpoint, whose cluster gateway distinguishes replicas by path. default: /v1 minLength: 1 maxLength: 512 x-kubernetes-validations: - rule: "self.startsWith('/')" message: spec.api.prefix must start with a slash. model: type: string description: >- The name this backend knows the model by, which a gateway rewrites the request's model to on the way out. Unset, the caller's model name passes through unchanged. A caller names a ModelService and gets back whichever model actually served: ask for ml-team/assistant and the response names the model that answered, such as Qwen/Qwen3-8B. minLength: 1 maxLength: 253 credential: type: object description: >- This backend's credential, which the gateway attaches on the way out. When the gateway authenticates callers, the caller's own key never reaches the backend. An endpoint whose credential is missing carries no traffic and reports EndpointReady=False. required: [method] x-kubernetes-validations: - rule: "has(self.apiKey) == (self.method == 'APIKey')" message: spec.credential.apiKey must be set when spec.credential.method is APIKey, and only then. properties: method: type: string description: >- How the gateway authenticates to this backend. APIKey sends a key held in a Secret, in the x-api-key header to a backend whose api.schema is Anthropic, and as a bearer token in the Authorization header otherwise. enum: [APIKey] apiKey: type: object description: >- Authenticates to the backend with an API key. Required when method is APIKey. required: [secretRef] properties: secretRef: type: object description: >- The Secret holding the API key, in this ModelEndpoint's namespace. required: [name] properties: name: type: string minLength: 1 maxLength: 253 key: type: string description: The Secret key holding the API key. default: apiKey minLength: 1 maxLength: 253 status: type: object properties: conditions: type: array items: type: object ```