The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

Bloomberg Offers Guidance on Getting Data Annotation Right for Machine Learning

Subscribe to our newsletter

Machine learning has become essential to financial institutions seeking timely business insight and signals of opportunity and risk across the business. At many firms, the technology is being scaled and use cases are proliferating. There are limitations, however, with useful outcomes from machine learning models depending on high quality data that is annotated accurately and consistently.

Data annotation probably isn’t the first thing that comes to mind when considering machine learning projects, but it is crucial to success and often difficult to achieve. With this in mind, Bloomberg has pulled together its expertise in annotation and published it for the use of other organisations.

The publication, Best Practices for Managing Data Annotation Projects, provides a practical guide to planning, executing, and evaluating the annotation step in machine learning projects. It was authored by Amanda Stent, natural language processing (NLP) architect in the office of the CTO; Tina Tseng, legal analyst with Bloomberg Law; and Domenic Maida, chief data officer, global data.

Key considerations of data annotation covered by the publication include, how to:

  • Identify stakeholders that should be involved in a project
  • Decide on datasets to be included in the project
  • Write and share annotation guidelines
  • Select an annotation tool
  • Test annotation for correct results and edge cases
  • Select the right team for each project based on the data
  • Ensure consistent communication across the team
  • Manage time and budget to ensure all project data is covered
  • Evaluate annotation quality at the end of the project.

The authors note that data annotation projects are ongoing processes rather than one-off tasks, and acknowledge the need for a human in the loop ‘as we have more contextual value than computers’.

Bloomberg’s expertise in annotation is built on the need to understand different types and formats of data that flow through its data pipelines and analytics, including earnings releases and tables, PDFs of filings, news articles, and ever-changing information about stocks, maturity dates of bonds, foreign exchange rates, and commodity prices. The company uses and contributes to the open source tool pybossa for data annotation.

Subscribe to our newsletter

Related content

WEBINAR

Recorded Webinar: Best practices for metadata management

Metadata has become central to financial firms as a means of enriching, discovering and effectively using both business and technical data. Its management is equally important, particularly in capital markets applications such as data lineage and data governance, which require a clear view of data and its provenance, and the ability to scale and support...

BLOG

Corporate Actions a Challenge? Join Next Week’s A-Team Group Webinar to Find Solutions

Demand for timely and accurate corporate actions data is increasing as the complexity and volumes of events continues to rise, and financial institutions acknowledge the costly gap between accurate corporate actions processing in real, or near-real, time and faulty processing caused by poor data and resulting in missed opportunities to optimise revenue. While many firms...

EVENT

FRTB Briefing Virtual (Redirected)

The FRTB Briefing Virtual will examine key focus areas and priorities for banks in 2020 and offer guidance, advice and expertise for tackling the challenges and pain points around implementing the FRTB regulation.

GUIDE

ESG Handbook 2021

A-Team Group’s ESG Handbook 2021 is a ‘must read’ for all capital markets participants, data vendors and solutions providers involved in Environmental, Social and Governance (ESG) investing and product development. It includes extensive coverage of all elements of ESG, from an initial definition and why ESG is important, to existing and emerging regulations, data challenges...