AI Evaluation Lead

Job Description

This cookie notice applies to the Workday Careers website A cookie are widely used for website’s functionality, or to make it operate more efficiently, as well as to provide information to the owners of the site. We use re-targeting (which is a cookie-based technology) that uses simple Javascript code to anonymously ‘follow’ our audience all over the Web. We use this technology to target individuals for advertising campaigns.To understand how the FCA collects and processes personal data please refer to the .have been set, visit or .To find information relating to other browsers, visit the browser developer’s website.To opt out of being tracked by Google Analytics across all websites, visit . Banner .We use re-targeting (which is a cookie-based technology) that uses simple Javascript code to anonymously ‘follow’ our audience all over the Web.# Accessibility Statement This accessibility statement applies to content published on thedomain and its sub-domains. It does not apply to content on or other domains.This website is run by the Financial Conduct Authority. We are committed to inclusive design and ensuring that our web content is accessible to all.For example, that means you should be able to: zoom in up to 300% without the text spilling off the screen navigate most of the website using just a keyboard navigate most of the website using speech recognition software listen to most of the website using a screen reader (including the most recent versions of JAWS, NVDA and VoiceOver)We’ve also ensured that the text is clear and easy to understand. has advice on making your device easier to use if you have a disability. How accessible this website is We know some parts of this website are not fully accessible: Checkboxes are not accessible using standard voice commands. Some radio buttons are not given group titles when tabbing to them using a screen reader. Edit combo fields with options lists can be difficult to navigate to with voice commands when the field is completed. The programmatic labelling of edit combo fields with options lists is not clear when using a screen reader. Feedback and contact information If you need information on this website in a different format contact us and tell us: the web address (URL) of the content your name and email address the format you need – for example large print, braille or audio Reporting accessibility problems with this website We’re always looking to improve the accessibility of this website. If you find any problems not listed on this page or think we’re not meeting accessibility requirements, contact us. Enforcement procedure The Equality and Human Rights Commission (EHRC) is responsible for enforcing the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018 (the ‘accessibility regulations’). If you’re not happy with how we respond to your complaint, . Technical information about this website’s accessibility The Financial Conduct Authority is committed to making its website accessible, in accordance with the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018. Compliance status This website is partially compliant with the AA standard, due to ‘the non-compliances’ listed below. Non-accessible content The content listed below is non-accessible for the following reasons. Non-compliance with the accessibility regulations Checkboxes are not accessible using standard voice commands such as “Click checkbox” and can only be accessed by using “Tab” commands. This fails WCAG 2.1 success criterion 1.3.1 ( Information & Relationships ). Some radio buttons are not given group titles when tabbing to them using a screen reader. This fails WCAG 2.1 success criterion 1.3.1 ( Information & Relationships ). Edit combo fields with options lists can be difficult to navigate to with voice commands when the field is completed. This fails WCAG 2.1 success criterion 1.3.1 ( Information & Relationships ). The programmatic labelling of edit combo fields with options lists is not clear when using a screen reader. This fails WCAG 2.1 success criterion 1.3.1 ( Information & Relationships ). What we’re doing to improve accessibility We are actively engaging with our site developers and independent accessibility professionals to address the issues above and improve overall accessibility with the eventual goal of attaining full WCAG AA compliance. Preparation of this Accessibility Statement This statement was prepared on 22nd July 2021. It was last reviewed on 22nd July 2021.This website was last tested on 02/02/2021. Testing was carried out by Blazie Engineering using the Website Accessibility Conformance Evaluation Methodology (WCAG-EM) approach to deciding on a sample of pages to test. Job Title: AI Evaluation Lead Division: Data, Technology and Innovation Department: AI Product Delivery Salary: National (Edinburgh and Leeds) ranging from £72,100 to £100,000 and London from £79,300 to £110,000 per annum (salary offered will be based on skills and experience) This role is graded as: Technical Specialist – Regulatory Your external recruitment contact is Benjamin via . Your internal recruitment contact is Lauren via Applications must be submitted through our online portal. Applications sent via social media or email will not be accepted. About the FCA and team We regulate financial services firms in the UK, to keep financial markets fair, thriving and effective. By joining us, you’ll play a key part in protecting consumers, driving economic growth, and shaping the future of UK finance services.The Data, Technology and Innovation (DTI) division enables the FCA to be a digital-first, data-led smart regulator by delivering a secure, agile, and cost-effective technology and data ecosystem that drives better decisions, transparency, and operational efficiency.Working alongside the wider AI Programme (which will continue to oversee/coordinate AI activity across the FCA), the department will partner with business leads to shape and deliver work in priority areas – Authorisations, SPC, EMO and Anti Money Laundering. Role responsibilities Define and own evaluation frameworks for GenAI outputs covering quality measures such as accuracy relevance robustness and hallucination rates Design curate and govern test datasets and benchmarks to ensure consistent model and solution performance assessment over time Support development of automated evaluation pipelines and operational reporting to embed assurance into delivery Identify assess and mitigate risks in model behaviour for example bias errors safety concerns and edge cases with clear escalation and control recommendations Manage delivery through well-defined work packages setting priorities operating standards and performance objectives including line management of Business Analysts Engage senior stakeholders to understand strategic priorities build a pipeline of scoped and prioritised projects and translate needs into analytics led solutions with clear business value Work in the public interest protecting 40 million UK consumers who rely on financial services and supporting long term economic growth from an industry contributing 12% of UK economic output Manage digital and data-led change by encouraging innovative experiments and working with senior stakeholders while empowering a diverse team to collaborate openly Skills required Minimum: Experience delivering analytics, data science and AI/ML initiatives, including defining success measures, evaluating model/product performance, and applying innovative problem-solving approaches Demonstrated experience leading people (line management,