Sereact logo

Website Documentation

Introduction

In this section, the frontend site of the Sereact Picking System. After setting up the system, user can use any website browser to access the UI system via the following URL:

  • URL: http://<ip_address>:<port>/
    • ip_address: The ip_address is the address of the executing machine, it can also be the localhost.
    • port: The port for calling API. The default value is 3000.
    • Example URL: http://localhost:3000/

Main Panel

Main Panel

The main panel includes the quick access to the following services:

  • Home: The Homepage of our website, include the features for sampling the objects, calling the grasp and checking the robot state.
  • Configuration: The Configuration page allows user to change the configuration the system components such as the robot, the cameras, or the bins.
  • Health: The Health page shows the status of the system processes, user can also restart the processes from here.
  • Digital I/O: User can activate/deactivate the IO port using this page.
  • Visualization:
  • Frames: The Frames page illustrate all the frames of the system.
  • Statistics: The Statistics page indicates the performance of the system by showing the number of picks and number of successful picks.
  • Debug:
  • PickGPT: The Prompt for our PickGPT feature.
  • Jogging: User can jog the robot in either Cartesian space (x,y,z,rx,ry,rz) or Joint Space (1,2,3,4,5,6) by pressing the up/down buttons in the Jogging panel.

Home page

Main Panel

The Home Page of the Capto API frontend provides intuitive panels for controlling and monitoring the robot's execution processes. Here's a breakdown of the key functionalities:

  • Execute Grasps

    This panel facilitates the execution of the pick and place process. Users can specify the following parameters:

    • Number of items: Define the quantity of items to be picked.
    • Select pick area: Choose the designated pick area.
    • Select pack area: Specify the pack area for placing items.
    • Execute: Initiate the picking process with the specified parameters.

  • Robot State

    The Robot State panel allows users to check the current state of the robot and restart it if necessary. This is particularly useful for ensuring the robot is in the desired operational state.

  • Controls

    In the Controls panel, users have granular control over the robot's operation. The available options include:

    • Homing: Move the robot to its homing position.
    • Demo Mode: Toggle the demo feature on or off. In demo mode, the robot continuously picks and places items between bins.
    • Hardware: Check the status of the robot's hardware operation.
    • Stop: Halt all ongoing robot tasks and return it to the homing position.
    • Speed: Set the robot's speed using a percentage scale.

  • Current Execution

    The Current Execution panel provides real-time updates on the status of the ongoing picking process. This information allows users to monitor the progress and status of the robot's current tasks.

Main Panel

Below the panels, there is an area for viewing the current camera view and manually sampling grasps. When the user chooses the pick area and place area, hits the Sample button, the image from the current camera will appear in this section, along with the respective grasp information. The user can then hit the Execute button to instruct the robot to grasp the specific object based on the sampled data.

Configuration Page

Health Page

Main Panel

The Health Page provides an overview of the status of both software processes and the underlying machine. Here's a breakdown of the key components:

Application Health

This section displays the status of various components within the Capto software system. Users have the option to restart each process individually by hitting the button next to each process.

  • bin: The process responsible for generating bin information.
  • kinematics: The kinematics server of the system.
  • robot_execution: The process that controls all the tasks performed by the robot.
  • cam_x: Processes operating the cameras.
  • calib: The calibration process.
  • backend: The backend process.
  • inference: The process responsible for generating object information from the images received from the camera processes.

System Health

This section provides an overview of the hardware machine's status, including the following metrics:

  • CPU usage: The percentage of CPU utilization.
  • CPU freq: The frequency of the current CPU.
  • RAM: The percentage of RAM usage.
  • GPU temp: The temperature of the GPU.
  • GPU load: The percentage of GPU utilization.
  • GPU Power: The power consumption of the GPU.
  • GPU Mem: The RAM usage on the GPU.

Monitoring these metrics gives users insights into the overall health and performance of both the software processes and the machine, facilitating effective troubleshooting and maintenance.

Digital I/O Page

Main Panel

The I/O Control Page serves as a centralized interface for managing external Input/Output (I/O) ports. Users can perform the following actions on this page:

  • Reading I/O Ports Status: Users can view the status of the digital input and output ports, providing real-time information about the state of external devices connected to the system.
  • Setting Output Ports: Users have the capability to set the state of Digital Output ports through this page. This feature allows for the control and manipulation of connected external devices.

Visualization Page

The Visualization Page provides a comprehensive view of the current world configuration, incorporating details such as the robot state and bin positions. Users can further enhance the visualization by utilizing the following buttons:

  • Show grid: Reveals the grid used in the path planning procedure of the robot, providing insights into the planned trajectory.
  • Show sanity: Displays the working range of the robot, allowing users to visualize the areas within which the robot can effectively operate.
  • Show frames: Exhibits all the frames in the world configuration, offering a detailed view of the spatial relationships between different components.
  • Show pointcloud: Presents the point cloud received from the selected camera, providing a visual representation of the environment as perceived by the robot. By utilizing these buttons, users can customize the information displayed on the Visualization Page, gaining a deeper understanding of the robot's current state and its interaction with the surrounding environment.
Main PanelMain Panel

Frame Page

Main Panel

The Frame Page provides essential information about crucial frames in our system. Users can conveniently read and copy this information using the buttons located next to each set of numbers. The frames include:

  • TCP Pose: The current 7D pose of the Tool Center Point (TCP) in world coordinates.
  • Calibration Target: The 7D pose of the calibration target in world coordinates.
  • Robot Flange: The 7D pose of the robot flange in world coordinates.
  • Joint State: The current robot joint state, represented in 6D coordinates. By utilizing the copy buttons next to the numerical values, users can easily extract and use this important frame information for various purposes within the Capto API system.

Statistics Page

The Statistic Page offers an overview of system performance statistics over its entire history. Users can manipulate the displayed graph using the buttons provided next to the graph. The three crucial data points include:

  • Total Picks: Represents the total number of pick operations performed in the last 7 days.
  • Pick per Day: Displays the average number of picks per day over the last 7 days.
  • Effectiveness: Illustrates the success rate by showcasing how many picks were successful compared to the total number of picks in the past 7 days. Users can interact with the graph using the provided buttons, allowing for a dynamic and customizable analysis of the system's historical performance.
Main Panel