The firm's lexicon-based review was also not reasonable. The firm contacted its email provider to discuss appropriate lexicon search terms and selected 24 search terms that would "flag" an email for a principal review. Collectively, these search terms were not comprehensive enough to yield a meaningful sample of flagged communications. Moreover, the lexicon was not based on an assessment of risk areas at the firm, nor was it reasonably tailored to the firm's size, structure and business model. As a result, most the search terms resulted in an unreasonably small number of emails flagged for review. Further, two search terms generated the vast majority of the flagged emails, and at least one of those terms was ineffective because it resulted in an unreasonably high percentage of "false positives." Despite the obvious indications that the firm's lexicon system was not reasonably designed, the firm did not evaluate the efficacy or make any changes to its lexicon system during the entire Relevant Period.

Lexicon-Based Reviews of Electronic Correspondence - Members using lexicon-based reviews (those based on sensitive words or phrases, the presence of which may signal problematic communications) of correspondence should utilize an appropriate lexicon, take reasonable security measures to keep the list confidential and periodically evaluate the efficacy of the lexicon. Members must make informed decisions regarding how best to utilize the surveillance tools they have chosen. Thus, a member that conducts lexicon-based reviews may determine that it is not necessary to review each and every lexicon "hit" in order to maintain an effective review system. The rationale for such determinations should be maintained as part of the member's policies and procedures. 

Members should also consider regular periodic reviews of the lexicon system to determine whether any changes/updates are necessary, such as adding or deleting phrases and/or words. Members should periodically inquire as to the effectiveness of the system, especially if the system is that of a vendor. Members are responsible for ensuring that the system utilized is functioning properly. As discussed more fully below, if a member does not have confidence in the effectiveness of its lexicon system, a supplemental random review of electronic communications should be considered. 

Members should consider targeted concentrated reviews of employees' emails when warranted (e.g., when concerns are raised in connection with regulatory examination findings, internal audits, customer complaints or regulatory inquiries). 

When assessing the effectiveness of a lexicon-based system, members should consider the following features: 

(a) A meaningful list of phrases and/or words (including industry "jargon") based on the size of the member, its type of business, its customer base and its location (including any branch offices that may require the inclusion of certain foreign language components). The lexicon system should be comprehensive enough to yield a meaningful sample of "flagged" communications. 

(b) Ability to add and delete phrases and words on an ongoing basis.

(c) Ability to review attachments and identify attachments that could circumvent lexicon-based reviews. 

(d) Ability to restrict access to the phrases and/or words that make up the lexicon system. 

(e) Ability to conduct searches that exclude any trailers or disclaimers used by the member, as these trailers or disclaimers often contain sensitive words such as "guarantee" (e.g., "firm does not guarantee") which would "flag" every such email. 

Random Review of Electronic Correspondence - Members may choose to use a reasonable percentage sampling technique, whereby some percentage of the electronic communications generated by the member is reviewed. There is no prescribed minimum or fixed percentage that is required by regulation. However, the amount of electronic communications chosen for review must be reasonable given the circumstances (for example, member size, nature of business, customer base and individual employee circumstances). In this regard, members conducting random reviews may consider factors such as: 

(a) Percentage of Electronic Correspondence Based on a Branch Office, Department or Business Unit - For a branch office, department or business unit, a member could establish a percentage of electronic communications requiring review that is based on its size, type of business, customer base and location (including its sales locations), which includes emails from each individual in that branch office, department or business unit. 

(b) Percentage of Electronic Correspondence for Each Individual - For each individual in a branch office, department or business unit, a member could establish a percentage of emails requiring review based on its size, type of business, supervisory structure (including whether certain locations are supervised remotely), customer base and location including its branch offices. Members should not necessarily limit themselves to reviewing the same percentage of emails for each employee. For example, an individual with disciplinary history or subject to special supervision may warrant a review encompassing a higher percentage of emails