Pick from a moving belt¶
Walks through examples/basics/sequence_demo.py
— a tracking pick on the factory cell, baked into one deterministic timeline.
The conveyor feeds a box down the belt until it interrupts a photoelectric beam at the pick point — and then keeps running. The sequence latches onto the box, so every pose taught at the station rides along with the part: the robot dives onto the moving box, closes on it in motion, and only lets go of the belt sync once the box is its own.
cycle time: 16.69s
0.00 – 6.01 feed
6.01 – 6.01 latch
6.01 – 6.61 descend
6.61 – 7.01 close
7.01 – 7.01 grasp
7.01 – 7.61 lift
7.61 – 11.84 carry
11.84 – 12.64 lower
12.64 – 12.64 release
12.64 – 13.04 open
13.04 – 13.84 retreat
13.84 – 14.34 settle
14.34 – 16.69 home
tracked pick: caught the box 150 mm downstream, belt still running
exported to cell_seq.usda — view with: usdview cell_seq.usda
Thirteen steps, and the number in the last line is the point: between the latch and the grasp the belt carried the box 150 mm. Without tracking, that is how far the pick would have missed by.
The infeed¶
The demo builds on the Pose and plan scene, which already
has a belt: conv is the catalog conveyor that cell was ordered with, and its
transport zone came with it. What this demo adds is a part at the head of the
queue and a beam to see it arrive:
# ---- the two taught stations, straight out of the USD cell ----------
pick = scene.frame("/World/Conveyor/PickFrame")
place = scene.frame("/World/Pallet/PlaceFrame")
# ---- conveyor feed: Box_A starts upstream, a beam guards the pick ---
# The taught grasp sits at the box's centre, so the pick frame's height
# is also where the box rides down the belt. The belt needs no declaring
# here: `conv` is the catalog conveyor demo.py ordered, and its transport
# zone came with it — sized to the belt, at the speed the drive is set
# to. All this step does is put the part at the head of the queue.
scene.set_obstacle_pose(BOX, (-0.9, pick[0][1], pick[0][2]))
# The beam trips once the box's leading face comes within the beam
# radius, so parking it half a box downstream of the pick frame fires
# the latch just as the box reaches the taught grasp — which is what
# makes tracking pick the box up rather than a fixed point in space.
trip_x = pick[0][0] + BOX_SIZE / 2 + BEAM_RADIUS
scene.add_beam_sensor(
"beam_pick",
frm=(trip_x, 0.42, pick[0][2]),
to=(trip_x, 0.82, pick[0][2]),
radius=BEAM_RADIUS,
watch=[BOX],
)
Note what is not here — no zone to size, no belt speed to restate. A standard part carries its own behavior, so a sequence only has to start it. What does need care is the beam: it is not placed at the pick frame but half a box-width plus a beam-radius downstream, because a beam trips when the box's leading face reaches it. Placed that way, the latch fires at the exact moment the box's center crosses the taught grasp.
Teaching, hover-first¶
# ---- teach the poses by IK posing (studio-equivalent workflow) ------
# Each station is solved hover-first, so the grasp warm-starts from the
# pose right above it and stays in the same posture family. Between the
# stations the robot goes back to the ready pose first: the pallet is a
# 150 deg base swing from the conveyor, and warm-starting across that
# walks the solver into a local minimum.
hover_q = teach_grasp(scene, pick, standoff=HOVER) # above the belt
grasp_q = teach_grasp(scene, pick) # pads around the box, still open
scene.set_joint_positions(home_q)
drop_q = teach_grasp(scene, place, standoff=HOVER) # above the pallet
place_q = teach_grasp(scene, place) # box resting on the crate
scene.set_joint_positions(home_q)
Each station is solved hover-first so the grasp warm-starts from the pose right above it and stays in the same posture family. Between the stations the robot returns to the ready pose first: the pallet is a 150° base swing from the conveyor, and warm-starting IK across that walks the solver into a local minimum. These two habits — hover-first, and re-seeding across big swings — carry to every cell you'll teach.
The finger stroke is chosen with the same care:
Closed is a millimetre a side into the 60 mm box — the cycle's only
by-design contact, which is why the finger pads are the only links allowed to
touch the carried box (touch_links). Open has to swallow the few millimetres
a joint-space ramp bows sideways on its way down; 0.04 is the joint limit,
which the planner excludes.
The sequence¶
# ---- the sequence ---------------------------------------------------
scene.define_signal("carrying")
ramp_to = lambda q: dict(zip(names, q)) # noqa: E731
sq = scene.sequence("pick_place")
# The belt starts and the robot pre-positions over the pick point at the
# same time; the step ends when the part has arrived *and* the arm is
# there to meet it (series contacts).
sq.step(
"feed",
actions=[bt.seq.start("conv"), bt.seq.motion("to_hover")],
transition=bt.seq.all_of(bt.seq.signal("beam_pick"), bt.seq.done()),
)
# No halt: from here the taught poses ride the box down the belt.
sq.step("latch", actions=[bt.seq.track(BOX)])
sq.step("descend", actions=[bt.seq.ramp(ramp_to(with_fingers(grasp_q, OPEN)), 0.6)])
sq.step("close", actions=[bt.seq.ramp({f: CLOSED for f in fingers}, 0.4)])
sq.step(
"grasp",
actions=[
# Grasping the tracked part freezes the sync offset, so the lift
# goes straight up from wherever the box was caught.
bt.seq.attach(BOX, link="/panda/panda_hand", touch_links=TOUCH),
bt.seq.set_signal("carrying"),
],
)
sq.step("lift", actions=[bt.seq.ramp(ramp_to(with_fingers(hover_q, CLOSED)), 0.6)])
sq.step("carry", actions=[bt.seq.untrack(), bt.seq.motion("to_pallet")])
sq.step("lower", actions=[bt.seq.ramp(ramp_to(with_fingers(place_q, CLOSED)), 0.8)])
sq.step(
"release",
actions=[bt.seq.detach(BOX), bt.seq.set_signal("carrying", False)],
)
sq.step("open", actions=[bt.seq.ramp({f: OPEN for f in fingers}, 0.4)])
sq.step("retreat", actions=[bt.seq.ramp(ramp_to(with_fingers(drop_q, OPEN)), 0.8)])
sq.step("settle", transition=bt.seq.elapsed(0.5))
sq.step("home", actions=[bt.seq.motion("home")])
return sq.name
Read it the way a PLC programmer would:
feedstarts the belt and the pre-position motion together, and its transition isall_of(signal, done)— series contacts. The step ends when the part has arrived and the arm is there to meet it.latchisbt.seq.track: from here, every commanded pose is carried by the box's motion since this instant. Note what is not here — nostop("conv"). The belt keeps running.descend/closeareramps— guarded, fixed-duration joint moves, the right tool for driving through contact where a collision-checked planner would refuse.graspattaches the box. Grasping the tracked part freezes the sync offset, so the followingliftgoes straight up from wherever the box was caught — not from where it was taught.carryunlatches and runs a planned transfer. Planned motions cannot run while tracking (they bake all their waypoints up front, and the target would run away from them); ramps can. Untrack first, then plan.settleis a timer (elapsed(0.5)), and the internalcarryingsignal brackets the transfer — both of which become assertable lanes in the timeline.
Ask the timeline¶
The step table above is timeline.step_spans. The 150 mm is two
object_pose queries:
# How far the belt carried the box between the latch and the grasp: the
# distance the pick would have missed by without tracking.
latch, grasp = spans["latch"][0], spans["grasp"][0]
travel = timeline.object_pose(BOX, grasp)[0][0] - timeline.object_pose(BOX, latch)[0][0]
print(f"tracked pick: caught the box {travel * 1e3:.0f} mm downstream, belt still running")
Anything the bake computed is queryable afterwards — that is what the next tutorial turns into a test suite.
The complete script¶
examples/basics/sequence_demo.py
"""PLC-style sequence demo: a tracking pick on the factory cell.
The conveyor feeds Box_A down the belt until it interrupts a photoelectric
beam at the pick point — and then keeps running. The sequence latches onto
the box (`bt.seq.track`), so every pose taught at the station rides along
with the part: the robot dives onto the moving box, closes on it in motion,
and only lets go of the belt sync once the box is its own. The rest is
structured like a real cell — *planned* transfer moves between stations,
*guarded* ramp moves (no collision check) for the approach/retreat through
contact, an internal `carrying` signal, and timer steps. Everything bakes
into one deterministic timeline (cycle time printed), then exports to USD.
The same scene also yields the cell's bill of materials: the belt, the rack
and the guarding came from the catalog and are already identified, the
photo-eye and the pedestal are *identified* here (`scene.set_part`), and
the parts list is derived, never typed.
Run with: python examples/basics/sequence_demo.py [out.usda]
"""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent))
import botrail as bt # noqa: E402
from demo import build_scene, teach_grasp # noqa: E402
BOX = "/World/Conveyor/Box_A"
BOX_SIZE = 0.06 # the carton in factory.usda, sized for the Franka hand
BEAM_RADIUS = 0.005
# Finger stroke: open wide enough to drop over the box, closed a millimetre
# a side into it. That squeeze is the cycle's only by-design contact, so the
# pads are the only links allowed to touch the carried box. The open width
# also has to swallow the few millimetres a joint-space ramp bows sideways
# on its way down (0.04 is the joint limit, which the planner excludes).
OPEN, CLOSED = 0.039, 0.029
TOUCH = ["/panda/panda_leftfinger", "/panda/panda_rightfinger"]
# Vertical standoff for the hover poses either side of a grasp.
HOVER = 0.15
def build_cycle(scene: bt.Scene) -> str:
"""Teaches the motions and the pick_place sequence; returns its name."""
names = scene.robot.joint_names
fingers = [n for n in names if "panda_finger_joint" in n]
home_q = list(scene.joint_positions)
# ---- the two taught stations, straight out of the USD cell ----------
pick = scene.frame("/World/Conveyor/PickFrame")
place = scene.frame("/World/Pallet/PlaceFrame")
# ---- conveyor feed: Box_A starts upstream, a beam guards the pick ---
# The taught grasp sits at the box's centre, so the pick frame's height
# is also where the box rides down the belt. The belt needs no declaring
# here: `conv` is the catalog conveyor demo.py ordered, and its transport
# zone came with it — sized to the belt, at the speed the drive is set
# to. All this step does is put the part at the head of the queue.
scene.set_obstacle_pose(BOX, (-0.9, pick[0][1], pick[0][2]))
# The beam trips once the box's leading face comes within the beam
# radius, so parking it half a box downstream of the pick frame fires
# the latch just as the box reaches the taught grasp — which is what
# makes tracking pick the box up rather than a fixed point in space.
trip_x = pick[0][0] + BOX_SIZE / 2 + BEAM_RADIUS
scene.add_beam_sensor(
"beam_pick",
frm=(trip_x, 0.42, pick[0][2]),
to=(trip_x, 0.82, pick[0][2]),
radius=BEAM_RADIUS,
watch=[BOX],
)
# ---- teach the poses by IK posing (studio-equivalent workflow) ------
# Each station is solved hover-first, so the grasp warm-starts from the
# pose right above it and stays in the same posture family. Between the
# stations the robot goes back to the ready pose first: the pallet is a
# 150 deg base swing from the conveyor, and warm-starting across that
# walks the solver into a local minimum.
hover_q = teach_grasp(scene, pick, standoff=HOVER) # above the belt
grasp_q = teach_grasp(scene, pick) # pads around the box, still open
scene.set_joint_positions(home_q)
drop_q = teach_grasp(scene, place, standoff=HOVER) # above the pallet
place_q = teach_grasp(scene, place) # box resting on the crate
scene.set_joint_positions(home_q)
def with_fingers(q: list, width: float) -> list:
"""The configuration with both finger joints set to `width`
(joint_names is in q-vector order)."""
q = list(q)
for f in fingers:
q[names.index(f)] = width
return q
# ---- planned transfer motions (fingers stay closed while carrying) --
scene.add_segment("to_hover", goal=with_fingers(hover_q, OPEN))
scene.add_segment("to_pallet", goal=with_fingers(drop_q, CLOSED))
scene.add_segment("home", goal=home_q)
# ---- the sequence ---------------------------------------------------
scene.define_signal("carrying")
ramp_to = lambda q: dict(zip(names, q)) # noqa: E731
sq = scene.sequence("pick_place")
# The belt starts and the robot pre-positions over the pick point at the
# same time; the step ends when the part has arrived *and* the arm is
# there to meet it (series contacts).
sq.step(
"feed",
actions=[bt.seq.start("conv"), bt.seq.motion("to_hover")],
transition=bt.seq.all_of(bt.seq.signal("beam_pick"), bt.seq.done()),
)
# No halt: from here the taught poses ride the box down the belt.
sq.step("latch", actions=[bt.seq.track(BOX)])
sq.step("descend", actions=[bt.seq.ramp(ramp_to(with_fingers(grasp_q, OPEN)), 0.6)])
sq.step("close", actions=[bt.seq.ramp({f: CLOSED for f in fingers}, 0.4)])
sq.step(
"grasp",
actions=[
# Grasping the tracked part freezes the sync offset, so the lift
# goes straight up from wherever the box was caught.
bt.seq.attach(BOX, link="/panda/panda_hand", touch_links=TOUCH),
bt.seq.set_signal("carrying"),
],
)
sq.step("lift", actions=[bt.seq.ramp(ramp_to(with_fingers(hover_q, CLOSED)), 0.6)])
sq.step("carry", actions=[bt.seq.untrack(), bt.seq.motion("to_pallet")])
sq.step("lower", actions=[bt.seq.ramp(ramp_to(with_fingers(place_q, CLOSED)), 0.8)])
sq.step(
"release",
actions=[bt.seq.detach(BOX), bt.seq.set_signal("carrying", False)],
)
sq.step("open", actions=[bt.seq.ramp({f: OPEN for f in fingers}, 0.4)])
sq.step("retreat", actions=[bt.seq.ramp(ramp_to(with_fingers(drop_q, OPEN)), 0.8)])
sq.step("settle", transition=bt.seq.elapsed(0.5))
sq.step("home", actions=[bt.seq.motion("home")])
return sq.name
def identify_parts(scene: bt.Scene) -> None:
"""Pins what the cell's equipment *is* — the identity the BOM is
derived from. The belt, the rack and the guarding arrive identified:
they came from the catalog, so their part numbers, masses and
configurations are already on them. What is left is everything that did
not — this robot is NVIDIA's Isaac USD, and the scenery of the factory
stage is geometry until a part is pinned to it: the pedestal (a whole
USD subtree), the pallet, the photo-eye. Free attributes (`mass_kg`)
are summed by `bom.total()`. The four lines below are the difference
between a bill you can send to a supplier and one with holes in it —
and the catalog lines are the ones nobody had to write."""
scene.set_part("panda", manufacturer="Franka Robotics", model="Panda", mass_kg=18)
scene.set_part("beam_pick", manufacturer="KEYENCE", model="PZ-G61N", category="sensor.photoelectric")
scene.set_part("/World/Pedestal", model="PD-500", category="structure.pedestal",
description="robot pedestal, 4 anchors", mass_kg=120)
scene.set_part("/World/Pallet", model="EPAL 1", category="pallet", mass_kg=25)
def main() -> None:
out = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("cell_seq.usda")
scene = build_scene()
name = build_cycle(scene)
identify_parts(scene)
timeline = scene.simulate_sequence(name)
print(f"cycle time: {timeline.duration:.2f}s")
spans = {step: (start, end) for step, start, end in timeline.step_spans}
for step, start, end in timeline.step_spans:
print(f" {start:6.2f} – {end:6.2f} {step}")
# How far the belt carried the box between the latch and the grasp: the
# distance the pick would have missed by without tracking.
latch, grasp = spans["latch"][0], spans["grasp"][0]
travel = timeline.object_pose(BOX, grasp)[0][0] - timeline.object_pose(BOX, latch)[0][0]
print(f"tracked pick: caught the box {travel * 1e3:.0f} mm downstream, belt still running")
warnings = timeline.export_usd(out, fps=60.0)
for w in warnings:
print(f"warning: {w}")
print(f"exported to {out} — view with: usdview {out}")
# The bill of materials falls out of the same scene: equipment lines
# (robot, conveyor, sensor) plus the scenery that was pinned as parts.
bom = scene.bom()
bom_path = out.with_name(out.stem + "_bom.csv")
bom.save(bom_path)
print(f"\n{bom.to_markdown()}")
print(f"BOM: {len(bom)} lines, {len(bom.unidentified())} still unidentified — written to {bom_path}")
if __name__ == "__main__":
main()
Next¶
- Turn a bake like this into CI assertions: Verify the cell in CI.
- Add a second arm to the same belt: Two arms, one belt.
- The exported
cell_seq.usdaplays in usdview as-is, or back in the studio: Export and replay USD.