Claude Skill

scenario-walkable-room

Use when one interior photo, uploaded or generated on Scenario, should become a room a visitor walks into in a web browser and plays with: props lifted out of the photo and rebuilt as 3D objects to grab, throw, and hear land, inside a Gaussian splat world, or restyled variants of

LLM Mart · 0 points · 0 0 0

Virus-scanned Reviewed automatically before listing.

Full trust report

Download scenario-labs-skills-skills_scenario-walkable-room-dc592af.zip · 9 KB

scenario-labs/skills 1035 129 forks MIT Updated 5h ago

Part of scenario-labs/skills — 97 skills

Install

skills CLI npx skills add https://github.com/scenario-labs/skills/tree/main/skills/scenario-walkable-room

Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install scenario-labs-skills@llmmart

Git git clone https://github.com/scenario-labs/skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole scenario-labs/skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Scenario Walkable Room

Overview

One photo of a room becomes a place to walk into and touch: a complete 3D Gaussian splat world (walls behind the camera included) with the room's small objects rebuilt as real 3D models a visitor can pick up, throw, and hear land. Restyle the photo first and the same room comes in several looks. The route follows the open-source image-blaster pipeline, with every generation on Scenario through MCP:

Uncover: describe the photo literally and list its single movable objects with a box, the surface each stands on, a size estimate, and materials. Pick two or three per room.
Plate: one removal-only image edit that erases those objects and nothing else.
Cut-outs: one image edit per object on the original photo, drawing it alone on white.
World: a single-image world model on the plate.
Objects: image-to-3D on each cut-out, with PBR textures.
Sounds: a few short impact one-shots per object.
Assembly: level and scale the world from its own splats, fit the lens it assumed, derive a collision heightfield, put each object back where the photo had it, and show it all with a streaming splat renderer and a physics engine.

Steps 1 and 7 are the agent's own work on downloaded files; every generation in between is an MCP call. Connection, uploads, and the core loop: see the scenario skill. Edits: scenario-image or scenario-gemini-image. Worlds: scenario-3d-worlds. 3D models: scenario-3d. Sound: scenario-audio or scenario-sonilo. If a sibling skill named here is missing from your available skills, ask the user to install it (npx skills add scenario-labs/skills --skill <name>); unattended, proceed from tool schemas and flag the gap.

Quick reference

Discover each paid stage's member with recommend, passing the capability and the user's own words, and read next_step before taking a pick, per the scenario skill. recommend has no capability for worlds, so find the single-image world member with search as scenario-3d-worlds teaches. Never assert a generative model's id as a constant. Read every pick's model_schema_get and price the exact payload with model_run dry_run=true before running it.

Step	Discovery	What to send and check
Uncover	none, read the photo yourself	Single movable items only, with a normalized box, support, size in meters, materials (references/objects.md)
Plate	`img2img`, takes a reference	`remove the following from the image: ...` and nothing else; compare with the photo for over-removal and recoloring
Cut-out	`img2img`, same member	One edit per object on the original photo, square, about 1K, white background (references/objects.md)
World	`search`, single-image world	The plate, full-resolution splats, a fixed `seed`, a prompt describing the room without the objects
Object	`img23d` with PBR	One cut-out per job, PBR on, a moderate face count (about 50,000); pick the GLB by its type (`asset_get` `mimeType` `model/gltf-binary`), never by output order
Sound	`txt2audio`, sound effects	One clip of about 6 s per object holding several impacts separated by silence
Calibrate	none, local	Floor, level, scale, lens, heightfield, placement (references/calibration.md)
Viewer	none, local	Paged splat streaming, heightfield walker, convex-hull props, grab and throw, panned sounds (references/viewer.md)

Launch every batch with wait=false in waves within the team's concurrency ceiling (the parallel-custom-jobs row in the scenario skill: a launch refused with that 429 created no job, so relaunch it in the next wave), then jobs_wait on the ids, re-called with pending_job_ids on timeout. A world takes several minutes; a timeout is not a failure and never justifies a second model_run.

Worked example: the user's living room, walkable, with things to throw

Photo. upload_asset the user's photo (per the scenario skill: file_size, PUT the parts, upload_asset_complete) and reuse the asset id. With no photo, recommend with capability="txt2img" for a photoreal interior: a doorway view at about chest height, level and centered, wide lens, a few small objects on tables and the sofa. For several looks, edit that photo once per style, opening the prompt with the fixed architecture (windows, door, floor plan, camera, lens) before the new finishes, furniture, and light.
Filing. Create the run's collection before the first generation (collection_create through the catalog write lane, arguments under parameters), then collection_add_assets each keeper as it lands.
Uncover. asset_display each photo, describe it literally, and write the object list. Choose two or three objects per room: separate, unoccluded, one or two materials, standing on the floor, a table, or a seat.
Plate. recommend with capability="img2img" and "remove objects from a room photo, keep everything else", model_schema_get for the reference-image field, dry_run, then run with the photo as the reference:

remove the following from the image: the on the , ; the

asset_display the plate next to the photo. Removal edits sometimes recolor an object instead of erasing it, or take its neighbors too (every cushion on a bench, a carved screen beside a lamp): re-run with a narrower list, or drop that object.
Cut-outs. One edit per object on the original photo, never the plate, with the isolation prompt in references/objects.md. Name anything resting on the object so it stays out.
World. Find the single-image world member per scenario-3d-worlds, model_schema_get, dry_run, then run each plate with wait=false, full-resolution splats, one fixed seed for every room, and a prompt describing the room as it is without the removed objects. Collect the ids with jobs_wait (re-called with pending_job_ids; a world takes several minutes), then asset_download each .spz.
Objects. recommend with capability="img23d" and "textured PBR prop from a product cut-out", read the face-count, texture, and PBR fields off model_schema_get, dry_run, then one job per cut-out with wait=false and jobs_wait on the ids. Each job returns several assets (the mesh with its textures and previews): take the one whose asset_get mimeType is model/gltf-binary, inspect it in the viewer (scenario-3d), then asset_download it.
Sounds. recommend with capability="txt2audio" and "short impact sound effects", then one clip per object:

Four separate one-shot impacts, each followed by a full second of silence: {the object, its material} {knocked over / dropped} onto {the room's floor}, . Close-miked, dry room, no music, no voices.
Assemble. Calibrate each world and place its objects per references/calibration.md, then build the page per references/viewer.md. Simplify the GLBs for the web (about 15,000 to 20,000 triangles, 1K WebP textures, Meshopt) and load physics after the room is on screen.

Common mistakes

Treating the world as metric: it is not. Measure the camera's height above the fitted floor and compare the objects' projected heights with their estimated sizes; at authoring time that put the photo camera about 1 m above the floor in every room.
Assuming every world shares the photo's lens: restyles of one photo came back with fields of view from about 50 to 76 degrees from the same world model. Fit the lens per world before projecting boxes.
Cutting out from the plate: the object is no longer there. Cut-outs come from the original photo.
Uncovering plants, glass, or objects holding other objects: image-to-3D closes foliage into a solid shell with leaves painted on. Skip them, or name and remove what they hold.
Placing objects from the base of the box on the floor plane alone: objects on tables or seats float or sink. The surface right behind the bottom of the box is where each one stood.
Dropping props into physics on page load: noisy supports send them tumbling. Start rigid props asleep where the photo had them; let only soft ones (cushions) settle.
Playing a sound on every contact force: a resting object reports its own weight every step. Sound only when the object was moving into the contact.
Trusting PBR metalness from image-to-3D: lacquer and glaze come back as metal and render as chrome. Keep metalness only on real metal.
Changing the payload after a 3D job reports failure with a result-download error: that is a platform error, not a bad input. Confirm with job_get that the job's status is failure and it has no assets (and that no later job for the same input succeeded, via jobs_list) before retrying once with the same payload; a jobs_wait timeout is not a failure and never justifies a second run.
Running heavy local jobs in parallel (calibration, mesh conversion, splat level-of-detail builds): run them one or two at a time.

Files (skills)

references

calibration.md 3.8 KB

# Calibrating a single-image world

A single-image world model on Scenario returns a Gaussian splat file (`.spz`) and nothing else: no metric scale, no floor, no collider. Everything below is derived from the splats, with the world's origin at the photo camera. Write it as a local script (numpy is enough) and run it one world at a time.

## Read the splats

SPZ is gzip. After the 16-byte header (magic `NGSP`, version, point count, spherical-harmonics degree, fractional bits) come all positions (3 × 24-bit fixed point), then opacities, colors, scales (`exp(byte / 16 - 10)`), and rotations. Positions are enough for calibration; keep opacity to drop faint splats. The worlds at authoring time used OpenCV axes (x right, y down, z forward); a 180° turn about x gives three.js axes (y up, camera looking down -z). Check the axes on the first world by finding the floor below the origin.

## Floor, level, scale

1. Fit a plane by RANSAC to opaque, small splats below the camera and within a few meters of it, then refine it with a least-squares fit on the inliers.
2. Rotate the world so the plane normal points straight up (one to two degrees of camera tilt is typical) and translate it so the floor sits at y = 0.
3. The camera's height above the plane in world units is not metric. Measure it, project each object's box into the world (next sections) at a trial scale, and compare its height with its estimated real size. The ratio is consistent across objects, and it gives the scale. Across seven rooms at authoring time the photo camera sat about 1.0 m above the floor (0.9 to 1.1 m), ceilings then came out at 2.7 to 3.3 m, and object heights matched their estimates within about 10 percent. Walking at that eye height keeps the opening view identical to the photo.

## Lens

Project the splats from the origin into an image the size of the source photo for a range of horizontal fields of view, and keep the one whose rendering correlates best with the photo (luminance, z-buffered). Restyles of one photo came back with fields of view from about 50° to 76° from the same world model, so fit every world.

## Collision heightfield

Top-down grid, 4 cm cells, 2 cm height levels:

- Floor evidence: any opaque splat within 5 cm of y = 0. Far floor is made of larger splats, so do not filter by size here.
- Height: the robust top of small, opaque splats between 4 cm and 1.75 m in the cell (the highest level with at least three splats at or above it), median-filtered over 3 × 3 cells.
- Walls: cells dense from knee height to above 1.75 m.
- Inside: flood-fill from a free cell near the camera over low cells that have floor. Furniture next to that floor stays; everything else becomes wall, which also closes doorways and the view through windows.

A walker treats cells taller than a step (about 14 cm) as blocked and slides along them; the physics engine uses the same grid as a heightfield collider, plus a ceiling.

## Putting objects back

The plate no longer shows the objects, so the surface right behind the bottom edge of an object's box is where it stood:

1. Render a depth buffer of the world from the origin with the fitted lens (nearest opaque splat per pixel).
2. Read the depth just above the bottom center of the box (median of the nearest half of a small window).
3. Unproject that pixel at that depth: it is the front of the object's footprint, at the height of its support. Move it back from the camera by half the object's width.
4. Scale the object so its height matches the box height at that depth, and turn it to face the camera.
5. If it stood on another object you lifted out, stack it on that object instead.

Write the result per room as data the viewer reads: the world transform (rotation, translation, scale), the fitted field of view, the heightfield grid, and each object's position, rotation, scale, and material class.

objects.md 2.8 KB

# Objects: uncover, plate, cut-out, 3D, sound

## Uncover rules

Describe the photo literally before choosing anything: walls, floor, furniture, light, what is outside the windows. Then list objects with:

- `name`, a precise visual `description`, `materials`, `support` (floor, table, sofa, shelf), a normalized `bbox` [x0, y0, x1, y1], a size estimate in meters, and notes on occlusion.
- Only single movable items: lamps, cushions, bowls, vases, lanterns, side tables, chairs. Never sofas, rugs, curtains, built-ins, or anything fixed to a wall.
- Prefer objects that are unoccluded, made of one or two materials, with a clear silhouette. Skip plants, glass, and objects holding other objects unless you remove those too.

Keep the list as JSON next to the photo: the box and size estimate drive the scale and placement steps in calibration.

## Plate prompt

Removal only, every object with its position, nothing else asked:

> remove the following from the image: the {object} on the {support}, {where}; the {object} {where}; ...

Check for over-removal (neighboring cushions, decorative screens) and for recoloring instead of removal. When a run fails either way, ask for two outputs if the schema allows it and keep the one that is right, or narrow the list.

## Cut-out prompt

On the original photo, square output at about 1K:

> Isolate the {object} from this image: {its description and where it stands}. Reproduce it exactly as shown -- same colors, materials, and proportions. White background, centered, tight crop, studio lighting. No other objects, no scene, no people, no text, no shadows on the ground. Isolate the object and remove all clustered, adjacent, overlapping, or items resting on the target object{, including ...}. Create a clean render of that one single object that is true to the source image.

Naming what rests on the object ("including the brass lantern standing on its top") keeps it out of the cut-out.

## 3D

Image-to-3D with textures and PBR, a moderate face count (about 50,000 is plenty for a prop seen from a few meters). Then for the web: simplify to about 15,000 to 20,000 triangles (keep more for fretwork), 1,024 px WebP textures, Meshopt geometry (glTF Transform does all three): tens of megabytes become a few hundred kilobytes. Set metalness to zero on anything that is not real metal.

## Sound

One clip per object holding several separate impacts:

> Four separate one-shot impacts, each followed by a full second of silence: {object and material} {knocked over / dropped} onto {floor}, {character}. Close-miked, dry room, no music, no voices.

Split it on silence (for example ffmpeg `silencedetect` at -38 dB with gaps of 0.2 s), keep two to four hits, add a short fade-out, and normalize loudness. In the viewer, play one at random on impact, never the same twice in a row, panned toward where it landed.

viewer.md 3 KB

# Viewer: streaming splats, walking, physics

The setup below is the one that shipped, built on three.js with [Spark](https://sparkjs.dev) for splats and [Rapier](https://rapier.rs) for physics. Other renderers work if they stream splats by level of detail; the physics rules hold for any engine.

## Splats

- Convert each `.spz` to a paged level-of-detail tree with Spark's `build-lod` tool (`--quality --rad-chunked --max-sh=0`), built from the same Spark release as the renderer the page ships. A room of 1.6 to 1.8 million splats became a header plus about 40 chunks of roughly 1 MB, so coarse levels arrive first and detail follows where the visitor looks. Offer the full `.spz` as a download.
- Add one `SparkRenderer` to the scene with `enableLod: true` (`lodSplatScale` below 1 on phones), and load each room as `new SplatMesh({ url, paged: true })` with an absolute URL: chunk files resolve against the page, not a `<base>` element. Apply the calibrated world transform to the mesh and await `mesh.initialized`.
- When switching rooms, keep the incoming mesh in the scene but hidden until it has streamed enough splats (a few hundred thousand on desktop, or a timeout), then reveal it.
- Render on demand: request frames from Spark's `onDirty` callback while pages stream, and stop when nothing moves.

## Walking

The visitor is a small circle (about 24 cm) at the calibrated eye height that slides along the heightfield's blocked cells. Offer keys, a touch stick, and drag-to-look.

## Physics

- Load the physics engine after the room is on screen.
- Ground: the heightfield as a heightfield collider (blocked cells raised to wall height), a ceiling, and a safety floor just below y = 0 so nothing falls out.
- Props: one dynamic body per object with a convex-hull collider from its simplified mesh (a box if the hull fails), density, friction, and restitution by material class, light damping, and continuous collision detection on.
- Rigid props start asleep exactly where the photo had them and wake on touch; soft ones (cushions) start awake and settle onto their seats.
- The visitor is a kinematic capsule that shoves whatever it walks into.
- Grab and throw: on pointer down, joint the hit object with a spherical joint to a kinematic anchor that follows the pointer at the grab depth, and raise its angular damping while held. On release, remove the joint and set the object's velocity to the pointer's recent velocity, clamped. A tap without a drag nudges the object with a small impulse instead.
- Impact sounds: enable contact-force events with a threshold scaled by the object's density, record each object's speed before the physics step, and play a sound only when that speed was meaningful (about 0.45 m/s), the object is not held, and no sound played on it in the last 150 ms. Stay silent for the first second or two while soft props settle. Pan and attenuate from the contact point.
- Lighting for the props: a neutral studio environment map, a hemisphere light, and a sun tinted by each room's palette, so the meshes sit in the splat room instead of glowing on it.

SKILL.md 10.3 KB

---
name: scenario-walkable-room
description: "Use when one interior photo, uploaded or generated on Scenario, should become a room a visitor walks into in a web browser and plays with: props lifted out of the photo and rebuilt as 3D objects to grab, throw, and hear land, inside a Gaussian splat world, or restyled variants of one room. Keywords: image-blaster, walkable room, photo to 3D room, clean plate, object cut-out, splat world, physics props, grab and throw, impact sounds, heightfield collision."
license: MIT
---

# Scenario Walkable Room

## Overview

One photo of a room becomes a place to walk into and touch: a complete 3D Gaussian splat world (walls behind the camera included) with the room's small objects rebuilt as real 3D models a visitor can pick up, throw, and hear land. Restyle the photo first and the same room comes in several looks. The route follows the open-source [image-blaster](https://github.com/neilsonnn/image-blaster) pipeline, with every generation on Scenario through MCP:

1. **Uncover**: describe the photo literally and list its single movable objects with a box, the surface each stands on, a size estimate, and materials. Pick two or three per room.
2. **Plate**: one removal-only image edit that erases those objects and nothing else.
3. **Cut-outs**: one image edit per object on the original photo, drawing it alone on white.
4. **World**: a single-image world model on the plate.
5. **Objects**: image-to-3D on each cut-out, with PBR textures.
6. **Sounds**: a few short impact one-shots per object.
7. **Assembly**: level and scale the world from its own splats, fit the lens it assumed, derive a collision heightfield, put each object back where the photo had it, and show it all with a streaming splat renderer and a physics engine.

Steps 1 and 7 are the agent's own work on downloaded files; every generation in between is an MCP call. Connection, uploads, and the core loop: see the `scenario` skill. Edits: `scenario-image` or `scenario-gemini-image`. Worlds: `scenario-3d-worlds`. 3D models: `scenario-3d`. Sound: `scenario-audio` or `scenario-sonilo`. If a sibling skill named here is missing from your available skills, ask the user to install it (`npx skills add scenario-labs/skills --skill <name>`); unattended, proceed from tool schemas and flag the gap.

## Quick reference

Discover each paid stage's member with `recommend`, passing the capability and the user's own words, and read `next_step` before taking a pick, per the `scenario` skill. `recommend` has no capability for worlds, so find the single-image world member with `search` as `scenario-3d-worlds` teaches. Never assert a generative model's id as a constant. Read every pick's `model_schema_get` and price the exact payload with `model_run` `dry_run=true` before running it.

| Step      | Discovery                     | What to send and check                                                                                                                                          |
| --------- | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Uncover   | none, read the photo yourself | Single movable items only, with a normalized box, support, size in meters, materials ([references/objects.md](references/objects.md))                           |
| Plate     | `img2img`, takes a reference  | `remove the following from the image: ...` and nothing else; compare with the photo for over-removal and recoloring                                             |
| Cut-out   | `img2img`, same member        | One edit per object on the original photo, square, about 1K, white background ([references/objects.md](references/objects.md))                                  |
| World     | `search`, single-image world  | The plate, full-resolution splats, a fixed `seed`, a prompt describing the room without the objects                                                             |
| Object    | `img23d` with PBR             | One cut-out per job, PBR on, a moderate face count (about 50,000); pick the GLB by its type (`asset_get` `mimeType` `model/gltf-binary`), never by output order |
| Sound     | `txt2audio`, sound effects    | One clip of about 6 s per object holding several impacts separated by silence                                                                                   |
| Calibrate | none, local                   | Floor, level, scale, lens, heightfield, placement ([references/calibration.md](references/calibration.md))                                                      |
| Viewer    | none, local                   | Paged splat streaming, heightfield walker, convex-hull props, grab and throw, panned sounds ([references/viewer.md](references/viewer.md))                      |

Launch every batch with `wait=false` in waves within the team's concurrency ceiling (the `parallel-custom-jobs` row in the `scenario` skill: a launch refused with that 429 created no job, so relaunch it in the next wave), then `jobs_wait` on the ids, re-called with `pending_job_ids` on timeout. A world takes several minutes; a timeout is not a failure and never justifies a second `model_run`.

## Worked example: the user's living room, walkable, with things to throw

1. **Photo.** `upload_asset` the user's photo (per the `scenario` skill: `file_size`, PUT the parts, `upload_asset_complete`) and reuse the asset id. With no photo, `recommend` with `capability="txt2img"` for a photoreal interior: a doorway view at about chest height, level and centered, wide lens, a few small objects on tables and the sofa. For several looks, edit that photo once per style, opening the prompt with the fixed architecture (windows, door, floor plan, camera, lens) before the new finishes, furniture, and light.
2. **Filing.** Create the run's collection before the first generation (`collection_create` through the catalog write lane, arguments under `parameters`), then `collection_add_assets` each keeper as it lands.
3. **Uncover.** `asset_display` each photo, describe it literally, and write the object list. Choose two or three objects per room: separate, unoccluded, one or two materials, standing on the floor, a table, or a seat.
4. **Plate.** `recommend` with `capability="img2img"` and "remove objects from a room photo, keep everything else", `model_schema_get` for the reference-image field, `dry_run`, then run with the photo as the reference:

   > remove the following from the image: the {object} on the {support}, {where}; the {object} {where}

   `asset_display` the plate next to the photo. Removal edits sometimes recolor an object instead of erasing it, or take its neighbors too (every cushion on a bench, a carved screen beside a lamp): re-run with a narrower list, or drop that object.

5. **Cut-outs.** One edit per object on the original photo, never the plate, with the isolation prompt in [references/objects.md](references/objects.md). Name anything resting on the object so it stays out.
6. **World.** Find the single-image world member per `scenario-3d-worlds`, `model_schema_get`, `dry_run`, then run each plate with `wait=false`, full-resolution splats, one fixed `seed` for every room, and a prompt describing the room as it is without the removed objects. Collect the ids with `jobs_wait` (re-called with `pending_job_ids`; a world takes several minutes), then `asset_download` each `.spz`.
7. **Objects.** `recommend` with `capability="img23d"` and "textured PBR prop from a product cut-out", read the face-count, texture, and PBR fields off `model_schema_get`, `dry_run`, then one job per cut-out with `wait=false` and `jobs_wait` on the ids. Each job returns several assets (the mesh with its textures and previews): take the one whose `asset_get` `mimeType` is `model/gltf-binary`, inspect it in the viewer (`scenario-3d`), then `asset_download` it.
8. **Sounds.** `recommend` with `capability="txt2audio"` and "short impact sound effects", then one clip per object:

   > Four separate one-shot impacts, each followed by a full second of silence: {the object, its material} {knocked over / dropped} onto {the room's floor}, {the character of the sound}. Close-miked, dry room, no music, no voices.

9. **Assemble.** Calibrate each world and place its objects per [references/calibration.md](references/calibration.md), then build the page per [references/viewer.md](references/viewer.md). Simplify the GLBs for the web (about 15,000 to 20,000 triangles, 1K WebP textures, Meshopt) and load physics after the room is on screen.

## Common mistakes

- Treating the world as metric: it is not. Measure the camera's height above the fitted floor and compare the objects' projected heights with their estimated sizes; at authoring time that put the photo camera about 1 m above the floor in every room.
- Assuming every world shares the photo's lens: restyles of one photo came back with fields of view from about 50 to 76 degrees from the same world model. Fit the lens per world before projecting boxes.
- Cutting out from the plate: the object is no longer there. Cut-outs come from the original photo.
- Uncovering plants, glass, or objects holding other objects: image-to-3D closes foliage into a solid shell with leaves painted on. Skip them, or name and remove what they hold.
- Placing objects from the base of the box on the floor plane alone: objects on tables or seats float or sink. The surface right behind the bottom of the box is where each one stood.
- Dropping props into physics on page load: noisy supports send them tumbling. Start rigid props asleep where the photo had them; let only soft ones (cushions) settle.
- Playing a sound on every contact force: a resting object reports its own weight every step. Sound only when the object was moving into the contact.
- Trusting PBR metalness from image-to-3D: lacquer and glaze come back as metal and render as chrome. Keep metalness only on real metal.
- Changing the payload after a 3D job reports `failure` with a result-download error: that is a platform error, not a bad input. Confirm with `job_get` that the job's status is `failure` and it has no assets (and that no later job for the same input succeeded, via `jobs_list`) before retrying once with the same payload; a `jobs_wait` timeout is not a failure and never justifies a second run.
- Running heavy local jobs in parallel (calibration, mesh conversion, splat level-of-detail builds): run them one or two at a time.