Updates on NYC AI Employment-Decision Tools

October 17, 2022

By: Corinne E. Tierney

Late last year, the New York City Council passed Local Law 144, which regulates employers and employment agencies’ use of “automated employment decision tools,” (AEDT), in making employment decisions. This new law is set to take effect on Jan. 1, 2023. In summary, the new law prohibits an employer or employment agency from using automated employment decision tools in making employment decisions unless, prior to using the tool, the following requirements are met: (1) the tool has been subject to a bias audit within the last year; and (2) a summary of the results of the most recent bias audit and distribution data for the tool have been made publicly available on the employer or employment agency’s website. Please see our prior blog post for a more thorough summary of the law.

On Sept. 23, 2022, New York City’s Department of Consumer and Worker Protection (DCWP) proposed new rules to accompany and clarify the law, and there is a public hearing on these proposals currently scheduled for October 24, 2022 at 11:00 a.m.[i]

Clarifying Definitions. The newly proposed rules clarify several key terms which are noticeably silent on the face of the law. Some of the biggest takeaway definitions are explained more below.

• Independent Auditor: The law defined the term “bias audit” to mean an “impartial evaluation by an independent auditor.” The newly proposed rules clarify that an independent auditor must be “a person or group that is not involved in using or developing an AEDT.” Effectively, this means that a bias audit cannot be performed by the employer or the vendor/developer of the utilized AEDT.
• AEDT: The definition of AEDT is also silent in the original text of the law. However, these newly proposed rules clarify that the law should only apply to: (i) an exclusively relied on “simplified output” (such as a score, tag, classification or ranking), with no other factors considered; (ii) to use a simplified output as one of a set of criteria where the output is weighted more than any other criterion in the set; or (iii) to use a simplified output to overrule or modify conclusions derived from other factors including human decision making.
• Candidate for Employment: Candidate for employment is now defined as “a person who has applied for a specific employment position by submitting the necessary information and/or items in the format required by the employer or employment agency.”
• Screen: The term screen has been clarified to mean “to make a determination about whether someone should be selected or advanced in the hiring or promotion process.”
• Selection Rate: The selection rate produced by the AEDT means “the rate at which individuals in a category are either selected to move forward in the hiring process or assigned as a classification by an AEDT. Such rate may be calculated by dividing the number of individuals in the category moving forward or assigned a classification by the total number of individuals in the category who applied for a position or were considered for promotion.” For example, if 100 Hispanic women apply for a position and 40 are selected for an interview after use of an AEDT, the selection rate for Hispanic women is 40/100 (or 40%).
• Impact Ratio: Impact ratio is now defined to mean “either (1) the selection rate for a category divided by the selection rate of the most selected category or (2) the average score of all individuals in a category divided by the average score of individuals in the highest scoring category.” Such ratios would look like one of the below calculations:

Selection Rate for a Category
Selection Rate of the Most Selected Category

OR

Average Score of Individuals in a Category
Average Score of Individuals in the Highest Scoring Category

• Machine learning, statistical modelling, data analytics, or artificial intelligence: “Machine learning, statistical modelling, data analytics, or artificial intelligence” has now been clarified to mean “a group of mathematical, computer-based techniques: (i) that generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate’s fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; (ii) for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and other parameters for the models in order to improve the accuracy of the prediction or classification; and (iii) for which the inputs and parameters are refined through cross-validation or by using training and testing data.”

Bias Audit Requirements. A “bias audit” is defined as “an impartial evaluation by an independent auditor,” which includes “the testing of an automated employment decision tool to assess the tool’s disparate impact on persons of any component 1 category required to be reported by employers pursuant to” 42 U.S.C. § 2000e-8(c) and 29 C.F.R. § 1602.7. The proposals make it clearer that to satisfy the new law, a bias audit must include: (1) a calculation of the selection rate for each race/ethnicity and a sex category and (2) a comparison of such selection rates as to the most selected category to determine an impact rate using one of the above defined formulas.

Candidate Notice Requirements and Alternative Process/Accommodation. Local Law 144 requires that employers provide at least 10 business days’ notice to candidates and employees who reside in New York City that an AEDT will be used in connection with a given assessment or decision. The newly proposed rules would allow this notice requirement to be satisfied by either: (a) posting such notice on the jobs or careers section of the employer’s website, including in a job posting; or (b) by sending to candidates for employment by U.S. mail or e-mail. Employers should be aware that this notice must also include instructions “for how to request an alternative selection process or accommodation.” However, the proposals still fail to provide any definition of “alternative selection process or accommodation,” or help explain what such alternatives might look like in practice.

Published Results Notice. Under Local Law 144, prior to the use of an AEDT, employers and employment agencies in New York City are required make the following “publicly available on the careers or jobs section of their website in a clear and conspicuous manner”: (1) the date of the most recent bias audit of such AEDT and a summary of the results, which shall include the selection rates and impact ratios for all categories; and (2) the distribution date of the AEDT to which such bias audit applies. The newly proposed rules clarify that this requirement can be met with “an active hyperlink to a website containing the required summary of results and distribution date, provided that such link is clearly identified as a link to results of the bias audit.” Finally, employers and employment agencies must keep the summary of results and distribution date posted for at least six months after last using such AEDT for an employment decision.

Other Regulations on Automated Employment Decision Tools. In addition to the legislation in Illinois and Maryland as discussed in our last blog post, the White House recently released a comprehensive “Blueprint for an AI Bill of Rights,” which outlines several considerations which are vital to the use and implementation of AEDTs and artificial intelligence in the employment context. Such a step by the White House suggests that this legislation will continue to grow and expand at the city, state and federal levels and we look forward to furthering developments here at Bond.

If you would like any assistance in preparing your organization for compliance with the New York City law on automated employment decision tools, please contact Corinne Tierney, any attorney in our labor and employment practice or the attorney at the firm with whom you are regularly in contact.

[i] For more information on how to attend the public hearing, please click here, where a link to join the public hearing by videoconference or by phone is available Prior to the hearing, any individual or group may submit comments on the Proposed Rules. Written comments can be submitted in advance by email to Rulecomments@dcwp.nyc.gov or online. Those wishing to speak at the hearing will be allotted three (3) minutes of speaking time and must sign up in advance by calling (212) 436-0396.