Introducing AVS, an Open Standard for Autonomous Vehicle Visualization from Uber
AVS介绍，Uber的一种用于自动驾驶车辆可视化的开放标准
Xiaoji Chen, Joseph Lisee, Tim Wojtaszek, and Abhishek Gupta February 19, 2019
Understanding what autonomous vehicles perceive as they navigate urban environments is essential to developing the systems that will make them operate safely. And, just as we have standards for street signs and traffic infrastructure to help human drivers, autonomous vehicle developers would be well-served by a standard visualization platform to represent input from sensors, image classification, motion inference, and all other techniques used to build an accurate image of the immediate environment.
了解无人驾驶汽车在城市环境中行驶时的感受，对于开发使它们安全运行的系统至关重要。而且，正如我们为街道标志和交通基础设施制定标准以帮助人类驾驶员一样，标准的可视化平台将为自动驾驶汽车开发商提供良好的服务，以表现来自传感器、图像分类、运动推断、以及用于构建即时环境准确图像的所有其他技术的输入。
As we've written before, the Advanced Technologies Group (ATG) and Visualization teams at Uber leverage web-based visualization technologies to interpret these sensor and algorithmic-derived worlds in order to support an ever-growing pool of autonomous use cases.
正如我们之前所写，Uber的高级技术小组（ATG）和可视化团队利用基于Web的可视化技术来解释这些传感器和算法派生的世界，以支持不断增长的自动驾驶用例池。
Today, we're excited to open source the redesigned and expanded Autonomous Visualization System(AVS), a new way for the industry to understand and share its data.
今天，我们很高兴能将经过重新设计和扩展的自动驾驶可视化系统（AVS）开源，这是业界了解和共享其数据的一种新方式。
AVS can display an autonomous vehicle's performance in the real-world.
AVS可以显示自动驾驶汽车在现实世界中的性能。
AVS is a new standard for describing and visualizing autonomous vehicle perception, motion, and planning data, offering a powerful web-based toolkit to build applications for exploring, interacting and, most critically, making important development decisions with that data.
AVS是用于描述和可视化自动驾驶车辆感知、运动和规划数据的新标准，它提供了一个功能强大的基于Web的工具包，可用于构建应用程序进行数据探索、数据交互，以及最重要的是可以使用该数据做出重要的开发决策。
As a stand-alone, standardized visualization layer, AVS frees developers from having to build custom visualization software for their autonomous vehicles. With AVS abstracting visualization, developers can focus on core autonomy capabilities for drive systems, remote assistance, mapping, and simulation.
作为独立的标准化可视化层，AVS使开发人员不必为自动驾驶汽车构建自定义可视化软件。借助AVS抽象可视化，开发人员可以专注于驾驶系统、远程协助、映射和仿真等核心自动驾驶功能。
The need for unified visualization
统一可视化的需求
A wide variety of organizations, including technology companies, foundations, research institutions, original equipment manufacturers (OEMs), and start-ups, are tackling the challenges of autonomous driving. Visualization tools, which display what autonomous vehicles perceive in their environments, are crucial for developing safe driving systems. The requirements for these tools generally originate close to the hardware and sensor stack and revolve around online or offline playback of autonomy system log data. As platforms mature, new use cases emerge around triage, simulation, mapping, safety and image collection, and labeling. The path to production requires a whole new layer of tooling and infrastructure around monitoring, remote assistance, and support.
各种各样的组织，包括技术公司、基金会、研究机构、原始设备制造商（OEM）和初创企业，都在应对自动驾驶的挑战。可以显示自动驾驶汽车在其环境中的感知能力的可视化工具，对于开发安全的驾驶系统至关重要。这些工具的要求通常起源于硬件和传感器堆栈，并且以自动驾驶系统日志数据的在线或离线回放为主要内容。随着平台的成熟，围绕分类、仿真、地图转换、安全性、图像收集和标记的新用例应运而生。通往产品化的道路上就需要能够围绕监控、远程协助和支持的全新的工具和基础架构层。
In addition to rapidly evolving requirements, autonomy engineers are often forced to learn complex computer graphics and data visualization techniques in order to deliver effective tooling solutions. The lack of a visualization standard has resulted in engineers assembling custom tools around ready-made technologies and frameworks in order to deliver solutions quickly. However, in our experience, these attempts at developing tools around disparate, off-the-shelf components lead to systems that are challenging to maintain, inflexible, and generally not cohesive enough to form a solid foundation for a platform.
除了快速变化的需求外，自动驾驶工程师经常被迫学习复杂的计算机图形和数据可视化技术，以便提供有效的工具解决方案。缺少可视化标准导致工程师围绕现成的技术和框架组装自定义工具，以便快速交付解决方案。但是，根据我们的经验，围绕分散的现成组件开发工具的这些尝试会导致系统难以维护，缺乏灵活性，而且通常缺乏能够成为一种平台坚实基础的足够的凝聚力。
We are sharing AVS with the broader autonomous community in the hope that collaboration across the industry will unlock more advancement, define a new standard, and lead to safer, more efficient transportation solutions for everyone.
我们正在与更广泛的自动驾驶社区共享AVS，希望跨行业的合作将释放更多的优势、定义新的标准，并为每个人提供更安全、更高效的运输解决方案。
Visualizing a world in motion
可视化一个运动中的世界
在Uber ATG基于Web的前后算法效果对比的AVS示例中，我们可以查看改进的车辆检测
In this example of AVS in Uber ATG's web-based before and after comparison application, we can view improved vehicle detection.
Autonomous vehicle development is a rapidly evolving area with new services, data sets (especially via LiDAR), and many use cases that require new solutions. At Uber, there were multiple engineering teams with unique requirements that our solution needed to address. Leveraging a web-based visualization application was an obvious choice as it created opportunities for fast iteration across teams, use-case specific applications, simplified information sharing, customization, and integration with existing services.
无人驾驶汽车开发是一个快速发展的领域，伴随着新的服务、数据集（尤其是通过LiDAR）和许多需要新解决方案的用例。在Uber，我们的解决方案需要满足有各自独特需求的多个工程团队。利用基于Web的可视化应用程序是一个显而易见的选择，因为它为团队之间的快速迭代、专属用例的应用、简化的信息共享、客制化、以及与现有服务的集成创造了机会。
While the benefits were clear, there were challenges in how to efficiently manage the data while retaining performance comparable to desktop-based systems. Solving for these challenges required a new abstraction to manage and describe the generated data that is used by web applications.
尽管好处显而易见，但如何有效管理数据，同时保持与基于桌面的系统相当的性能，仍然存在挑战。解决这些挑战需要一种新的抽象来管理和描述Web应用程序使用的生成数据。
Given the above requirements, we built our system around two key pieces: XVIZ provides the data (including management and specification) while streetscape.gl is the component toolkit to power web applications.
鉴于上述要求，我们围绕两个关键部分构建了我们的系统：XVIZ提供数据（包括管理和规范），而streetscape.gl是为Web应用程序提供支持的组件工具包。
XVIZ
We needed a formal and flexible specification for the data generated from autonomous systems, such that the data format could integrate with evolving infrastructure, conform across multiple clients, and be close enough to the source to define the necessary controls and bindings to efficiently manage it.
我们需要一个针对自动驾驶系统生成的数据的正式而灵活的规范，以便该数据格式可以与不断发展的基础架构集成，跨多个客户端兼容，并且与源头足够接近，以定义必要的控制和绑定来有效地对其进行管理。
XVIZ的高级数据流在服务器端结合了编码器和构建器，在客户端结合了解码器，数据缓冲区和同步器。
The high-level data flow for XVIZ incorporates an encoder and builder on the server side, with a decoder, data buffer, and synchronizer on the client side.
XVIZ provides a stream-oriented view of a scene changing over time and a declarative user interface display system. Like a video recording, you can randomly seek and understand the state of the world at that point. Like an HTML document, it's presentation is focused and structured according to a schema that allows for introspection. However, XVIZ also allows for easy exploration and interrogation of the data by tying together separate stream updates into a single object.
XVIZ提供了一个针对时变场景的面向流的视角，以及声明性的用户界面显示系统。就像录像一样，您可以随机地跳跃并且了解这个时间点上世界的状态。就像HTML文档一样，由于其允许内省的模型而使其具有专注的和结构化的呈现。不仅如此，通过将单独的流更新*绑捆**到单个对象中的方式，XVIZ还允许对数据进行轻松地浏览和查询。
An XVIZ stream is a series of discrete updates that occur at a specific time with specific primitive types. Primitives are objects that enable descriptions of information such as LiDAR point clouds, camera images, object bounds, trajectories, vehicle speed over time, and predicted plans. To simplify presentation for users, these objects can be individually styled (including at the stream level) or assigned a style class.
XVIZ流是一系列离散更新，它们在特定时间以特定的原始类型发生。原始对象是能够对信息进行描述的对象，例如LiDAR点云、摄像机图像、对象边界、轨迹、随时间推移的车辆速度以及预测的规划。为了简化向用户的呈现，可以对这些对象进行单独设置样式（包括在流级别）或分配样式类。
XVIZ organizes streams through hierarchical naming, with a separate metadata section listing the streams, their types, relative transforms, declarative UI panels, and style classes. The user interface then bundles graphical panels for the objects with the data, providing the user with the control via YAML to configure a series of layout and display components.
XVIZ通过分层命名来组织流，并在单独的元数据部分中列出了流、其类型、相对转换、声明性UI面板和样式类。然后，用户界面将对象的图形面板与数据*绑捆**在一起，从而通过YAML向用户提供控件以配置一系列布局和显示组件。
XVIZ's data structure lets us toggle streams from the dataset.
XVIZ的数据结构使我们能够切换数据集中的流
streetscape.gl
streetscape.gl提供了各种UI组件，包括摄像头、*放播**控件、对象标签和绘图功能。
streetscape.gl offers a variety of UI components, including camera, playback control, object label, and plot features.
streetscape.gl is a toolkit for building web applications that consume data in the XVIZ protocol. It offers drop-in-ready components for visualizing XVIZ streams in 3D viewports, charts, tables, videos, and more. It addresses common visualization pain points such as time synchronization across data streams, coordinate systems, cameras, dynamic styling, and interaction with 3D objects and cross components, so that users can invest more time building out autonomous vehicle software itself.
streetscape.gl是用于构建使用XVIZ协议数据的Web应用程序的工具包。它提供了用于在3D视窗、图表、表格、视频等中可视化XVIZ流的即用型组件。它解决了常见的可视化难题，例如跨数据流的时间同步、坐标系统、相机、动态样式以及与3D对象和跨组件的交互，从而用户可以投入更多时间来构建自动驾驶汽车软件本身。
Rendering performance is the top goal of streetscape.gl. Built on top of React and Uber's mature WebGL-powered visualization platform, we are able to support real-time playback and smooth interaction with scenes supporting hundreds of thousands of geometries.
渲染性能是streetscape.gl的首要目标。建立在React和Uber成熟的基于WebGL的可视化平台的基础之上，我们能够支持实时回放，以及与支持数十万个几何体的场景的平滑交互。
Composability is also front and center in the design of streetscape.gl. Learning from our work on our internal visualization platform, which powers a dozen diverse use cases such as triaging, labeling, debugging, remote assistance, and scene editing, we designed the components to be highly styleable and extensible, so that any team can build an experience that is tailored to their unique workflow.
可组合性在streetscape.gl的设计中也很重要。我们从内部可视化平台上的工作中学到了该知识，该平台支持分类、标记、调试、远程协助和场景编辑等十几种不同的用例，我们将组件设计为具有高度的可样式化和可扩展性，以便任何团队都可以按照他们独特的工作流程来构建一个量身定制的体验。
How AVS is different
AVS was designed to be open and modular, welcoming contributions from internal teams since the beginning of its development to enable decoupling. Architecturally, it provides a layered approach where the coupling between components of the autonomous stack is minimized and offers clear definitions for the exchange of data. Each layer can evolve as needed without requiring system-wide changes, and layers can be tailored for a specific context or us -case.
AVS有何不同
AVS设计为开放的和模块化的，自开发之初就欢迎内部团队的贡献，以实现去耦。从体系结构上讲，它提供了一种分层的方法，其中自动驾驶堆栈的各个组件之间的耦合被最小化，并为数据交换提供了清晰的定义。每个层都可以根据需要进行演变，而无需进行系统范围的更改，并且可以针对特定的上下文或我们的情况量身定制层。
This guiding principle helps set AVS apart from current solutions. Specifically, AVS's architecture distinguishes itself because:
- It was designed with an intentional separation of data from any underlying platform
- Its limited, small specifications make tools easier to develop
- Its data format requirements result in fast transfer and processing
该指导原则有助于使AVS与当前的一些解决方案区分开。具体来说，AVS体系结构的与众不同之处在于：
- 其设计目的是将数据与任何基础平台进行有意分离
- 其受限而小巧的规范使工具更易于开发
- 其数据格式的要求，使其能够快速传输和处理
Additionally, we created AVS to meet the needs of everyone in the autonomy ecosystem, including engineers, vehicle operators, analysts, and specialist developers. Autonomous engineers can describe their systems with XVIZ easily, and then test and visualize their expectations with limited overhead. Specialist developers can quickly build data source-agnostic applications with strong performance characteristics and simplified integration using streetscape.gl. Lastly, operators can view the data in standard visual formats, including videos, across multiple applications, leading to easier collaboration, knowledge understanding, deeper analysis, and overall trust in the data quality.
此外，我们创建了AVS，以满足自动驾驶生态系统中每个人的需求，包括工程师、车辆操作员、分析师和专业开发人员。自动驾驶工程师可以使用XVIZ轻松描述他们的系统，然后以有限的开销来测试和可视化他们的期望。专业开发人员可以使用streetscape.gl快速构建具有强大性能特征的与数据源无关的应用程序，并简化了集成过程。最后，操作员可以在多个应用程序中以标准的视觉格式（包括视频）查看数据，从而可以更轻松地进行协作、了解知识、进行更深入的分析以及对数据质量的整体信任。
By open sourcing it to the industry, we encourage more to contribute and build upon this initial set of ideas.
通过将其开源给行业，我们鼓励更多的人在这些最初的想法基础上做出贡献和建设。
Application in the industry and beyond
在行业及其他领域的应用
For companies building or supporting autonomous vehicles, such as Voyage,Applied Intuition, and Uber ATG, going from simulated or on-the-road tests to finding the root cause of issues can be an extremely time-consuming process.
对于建造或支持自动驾驶汽车的公司，例如Voyage，Applied Intuition和Uber ATG而言，从仿真或公路测试方式去查找问题的根本原因可能是一个非常耗时的过程。
According to Drew Gray, Voyage's CTO,being able to visually explore autonomous sensor data, predicted paths, tracked objects, and state information like acceleration and velocity is invaluable to the triage process and can positively impact developer efficiency. The information can then be used to set data-driven engineering priorities.
Voyage的CTO Drew Gray谈到，如果能够直观看到自动驾驶传感器数据、预测的路径、跟踪的对象、和像加速度和速度这样的状态信息，将对诊断过程非常有用，并能积极地促进开发人员的效率。然后，获得的信息可用于设置数据驱动的工程优先级。
Voyage co-founder Warren Ouyang echoes Gray's sentiment about the possibilities of AVS.
Voyage的联合创始人欧阳沃伦（Warren Ouyang）回应了Gray关于AVS可能性的观点。
"We're excited to use Uber's autonomous visualization system and collaborate on building better tools for the community going forward," says Ouyang.
欧阳说：“我们很高兴能够使用Uber的自动驾驶可视化系统，并协作为社区发展构建更好的工具。”
AVS在其他应用程序内部提供了丰富的上下文，例如此示例，它增强了Uber ATG的Spectacle事件查看应用程序。
AVS provides rich context inside other applications, such as this example where it enhances Uber ATG's Spectacle event review application.
Beyond root cause analysis, teams at Uber have also leveraged AVS for other use cases, such as web-based log-viewing, developer environments, and mapping maintenance. We also intend that, by open-sourcing the technology, developers in other nascent and adjacent industries, such as drones, robotics, trucking, fleet management, augmented and virtual reality, and retail, will find applications for this toolkit.
除了根本原因分析之外，Uber团队将AVS用于其他用例，例如基于Web的日志查看、开发人员环境和地图维护。我们还打算通过开源技术，在其他新兴行业和邻近行业的开发人员将找到该工具的应用，例如无人机、机器人技术、卡车运输、车队管理、增强和虚拟现实以及零售。
What's next
Bringing AVS to the broader industry is just the start. We envision it democratizing access for more developers and operators looking to contribute to the autonomous space.
将AVS带入更广泛的行业仅仅是个开始。我们的愿景是它将能够在更多希望为自动驾驶领域做出贡献的开发商和运营商中进行大众化普及。
In collaboration with partners like Voyage, Applied Intuition, contributors, and open source foundations, we plan to enhance the product with more data sources and specifications (especially ROS support), performance optimizations, and richer features such as side-by-side comparisons.
通过与Voyage、Applied Intuition、贡献者和开源基金会等合作伙伴的合作，我们计划通过更多数据源和规范（尤其是ROS支持）、性能优化以及更丰富的功能（如并排比较）来增强产品。
"At Applied Intuition, we're working with the most sophisticated AV teams in the world, and they require the most sophisticated tools," says Peter Ludwig, CTO of Applied Intuition. "AVS falls in line with this, and what's notably great is that it's web-based and fills a need in the community to not rebuild the same visualization tools again and again. This is an awesome move from Uber for the rest of the AV community."
“在Applied Intuition上，我们正在与世界上最先进的自动驾驶团队合作，并且他们需要最精巧的工具，”Applied Intuition的首席技术官彼得·路德维格（Peter Ludwig）说。“ AVS与此相符，最显著的是它是基于Web的，并且满足了社区的需求，即不必一次又一次地重建相同的可视化工具。对于其他自动驾驶社区来说，这是来自Uber的了不起的举动。”
Uber ATG's AVS-powered AV Log Viewer application lets us analyze a vehicle's approach to an intersection.
借助Uber ATG的AVS驱动的AV Log Viewer应用程序，我们可以分析车辆驶入交叉路口的方法。
Uber is interested in achieving a long-term vision of autonomous vehicles: a safer, cleaner, and more efficient transportation solution for everyone. Unfortunately, early developer tools in any industry tend to be primitive and adapted to solve new use cases that stretch their capabilities. Given how rapidly technology is transforming transportation and the cities we live in, the need for better tools to expedite this change is as urgent as ever.
Uber致力于实现自动驾驶汽车的长期愿景：为所有人提供更安全、更清洁、更高效的运输解决方案。不幸的是，任何行业中的早期开发人员工具通常都是原始的，并且已经适应解决扩展其功能的新用例。鉴于技术正在如此迅速地改变交通运输方式和我们所居住的城市，对能够加快这一变化的更好的工具的需求变得前所未有的迫切因此迫切。
Whether it's products that improve urban planning investments, richer geospatial analysis, advanced mapping, or new mobility trends, we find that an open data and tools strategy can help governments, developers, researchers, and the overall industry accelerate towards a smarter transportation ecosystem for the future.
无论是提高城市规划投资的产品、更丰富的地理空间分析、先进的地图绘制还是新的出行趋势，我们都发现开放的数据和工具策略将来可以帮助政府、开发商、研究人员和整个行业加速向更智能的交通生态系统发展。
If you're interested in developing technologies to advance the future of transportation, consider applying for a role at Uber. l If you're interested in contributing to AVS or other Uber open source projects, check out our open source program homepage.
如果您对开发技术以推动运输业的未来感兴趣，请考虑在Uber申请职位。l如果您有兴趣为AVS或其他Uber开源项目做出贡献，请查看我们的开源程序主页。
Special thanks to: Ib Green, Nicolas Belmonte, Xintong Xia, Travis Gorkin, Anthony Emberley, Jon Thomason, Carnaven Chiu, Jai Ranganathan, Jennifer Anderson, Neil Stegall, Charlie Waite, Ziyi Li and Erik Klimczak.
特别感谢：Ib Green，Nicolas Belmonte，Xintong Xia，Travis Gorkin，Anthony Emberley，Jon Thomason，Carnaven Chiu，Jai Ranganathan，Jennifer Anderson，Neil Stegall，Charlie Waite，Ziyi Li和Erik Klimczak。