Trusted Execution Environments & Confidential AI on NanoGPT

Trusted Execution Environments, or TEEs, are one of the most useful security tools available for hosted AI. They matter. We use them for that reason.

They also get overstated constantly.

This post is the narrow version: what NanoGPT TEE models actually give you, what they do not give you, and how to verify the parts that are verifiable.

Short version

A TEE can help with:

• Data in use: protecting model execution from the host OS, hypervisor, and ordinary infrastructure access
• Integrity: proving which measured code or environment handled the request
• Attestation: giving you provider evidence you can verify independently

A TEE does not, by itself, guarantee:

• end-to-end secrecy from your client all the way into the enclave
• zero logging across the provider stack
• that every proxy, gateway, or middleware layer is outside the trust boundary
• that every TEE provider exposes the same verification surface

That distinction matters. "TEE-backed" and "fully confidential end-to-end" are not the same thing.

What a TEE is

A Trusted Execution Environment is a hardware-isolated execution environment inside a CPU or GPU.

In practice, that usually means:

• code inside the enclave is isolated from the surrounding host environment
• enclave memory gets stronger protection than normal process memory
• the environment can produce an attestation report proving what was measured and where it ran

That is useful because hosted AI normally requires you to trust a large software and infrastructure stack. A TEE reduces that trust surface for the part of the request that is actually processed inside the enclave.

What a TEE does protect

For NanoGPT users, the strongest TEE benefits are:

1. Better protection of data in use

If the model inference is actually running inside a TEE, the host OS and ordinary admin access are much less useful for inspecting what is happening inside that measured environment.

That is meaningful. It raises the bar.

2. Better integrity guarantees

With attestation, you can verify that your request was handled by a specific measured environment rather than just taking the provider's word for it.

For some provider paths, you can also verify a signature tied to the completion itself.

3. A narrower trust boundary

Without a TEE, the default trust boundary is often "trust the whole provider stack." With a TEE, the trust boundary can be narrowed to "trust the attested enclave and the parts of the path outside it that still matter."

That is an improvement, even when it is not the same thing as end-to-end confidentiality.

What a TEE does not automatically protect

This is the part people usually skip.

1. It does not automatically give you HTTPS-to-enclave

Standard HTTPS terminates somewhere. Whether that termination point is itself protected by the TEE path depends on the provider architecture.

So if you care about prompt confidentiality, you need to ask:

• where TLS terminates
• whether any gateway or proxy can see plaintext
• whether provider SDKs or special transports are required for the strongest guarantees

2. It does not prevent logging by every layer outside the enclave

If a request goes through middleware, observability tooling, retry infrastructure, or a provider router before it reaches the enclave, those layers matter too.

TEE does not magically erase the rest of the pipeline.

3. It does not make transport or metadata concerns disappear

Even if the prompt body is well-protected, metadata and account-level context can still matter:

• IP address
• billing identifiers
• account activity
• prompt content that contains personal or company identifiers

If you put sensitive information directly into the prompt, a TEE does not make that risk vanish.

TEE vs other approaches

Here is the practical comparison:

Approach	Data in transit	Data in use	Integrity / attestation	Real-time practicality
Plain hosted inference	Usually yes	Weak	Weak	High
TEE-backed inference	Usually yes	Stronger	Stronger	High
End-to-end encrypted enclave path	Strongest	Stronger	Stronger	Provider-specific
Local models	You control it	You control it	You control it	Depends on hardware

For most hosted AI use cases, TEEs are one of the few options that improve security without destroying latency.

That still does not mean every TEE deployment has the same confidentiality properties.

How NanoGPT exposes TEE models

TEE-backed models are exposed with the TEE/ prefix.

Example:

curl -X POST https://nano-gpt.com/api/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "TEE/qwen-2.5-7b-instruct",
    "messages": [
      { "role": "user", "content": "Reply with exactly hello" }
    ]
  }'

How verification works on NanoGPT

The verification story is provider-specific.

Attestation

You can request attestation with:

curl -H "Authorization: Bearer <YOUR_API_KEY>" \
  "https://nano-gpt.com/api/v1/tee/attestation?model=TEE/llama3-3-70b"

Important caveat:

• some providers expose attestation documents directly
• some providers support nonce-bound freshness
• some do not
• some models may have upstream attestation temporarily unavailable

So treat attestation support as model- and provider-specific, not a universal property.

Completion-level signatures

Some RedPill-backed TEE paths expose a signature endpoint for the completion:

curl -H "Authorization: Bearer <YOUR_API_KEY>" \
  "https://nano-gpt.com/api/v1/tee/signature/<COMPLETION_ID>?model=TEE/qwen-2.5-7b-instruct"

The important detail is the identifier:

• use the chat response body id
• or use the X-TEE-Signature-ID header when present
• do not use NanoGPT X-Request-ID

NanoGPT non-stream TEE responses now expose:

• X-TEE-Completion-ID
• X-TEE-Signature-Available
• X-TEE-Signature-ID for supported RedPill-backed TEE models

That makes it easier to carry the correct identifier into verification tooling.

Current provider nuances

At a high level:

Tinfoil-backed TEE models

• attestation is available through the Tinfoil enclave attestation flow
• signature verification is not exposed through NanoGPT for these models
• nonce-bound freshness is not available through NanoGPT's attestation route

RedPill-backed TEE models

• some models expose completion-level signatures
• attestation behavior can vary by model and upstream availability
• if attestation is unavailable upstream, NanoGPT surfaces that clearly rather than pretending the verification exists

What we think the right claim is

The defensible claim is:

NanoGPT TEE models improve integrity and data-in-use protection, and may improve confidentiality depending on the provider path.

The claim we do not want to make is:

TEE means the provider stack can never see your prompts.

That second statement is too broad.

When you should use TEE models

TEE-backed models make sense when you want:

• stronger integrity guarantees than normal hosted inference
• a better story for sensitive prompts than ordinary multi-tenant inference
• attestation and, where supported, signature verification
• hosted performance instead of running your own local infrastructure

They make less sense if your requirement is:

• maximum secrecy against all third-party infrastructure
• no provider trust at all
• guaranteed end-to-end confidentiality regardless of gateway design

For that threat model, local models or infrastructure you control more directly are still the strongest option.

FAQ

Does TEE mean the provider cannot read my prompt?

Not automatically. It improves the data-in-use and integrity story, but the full confidentiality story still depends on transport, gateways, logging, and provider architecture.

Is TEE still worth it if it is not perfect?

Yes. "Not perfect" is not the same thing as "worthless." TEEs can materially improve the security posture of hosted inference.

What should I verify myself?

At minimum:

• the model route you are using
• whether attestation is available for that model
• whether signature verification is available
• which identifier the signature route expects

What is the safest wording for TEE claims?

Say that TEE improves integrity and data-in-use protection, and be explicit that transport and confidentiality guarantees are provider-specific.

Bottom line

TEE is useful. It is not magic.

If you want a hosted option that improves over ordinary provider trust, TEE-backed models are a real step forward.

If you want guarantees stronger than that, you need to look closely at the provider transport path, the verification surface, and in some cases whether hosted inference is the right trust model at all.