On October 1, 2018, a new Rule (specifically, a new subdivision to existing Rule 11-e) of the Commercial Division Rules, will go into effect. 

Rule 11-e governs Responses and Objections to Document Requests.  The new subdivision, promulgated by administrative Order of Chief Administrative Judge Lawrence K. Marks, governs the use of technology-assisted review (“TAR”) in the discovery process. 

The new subdivision (f) states:

The parties are encouraged to use the most efficient means to review documents, including electronically stored information (“ESI”), that is consistent with the parties’ disclosure obligations under Article 31 of the CPLR and proportional to the needs of the case.  Such means may include technology-assisted review, including predictive coding, in appropriate cases…

In addition to implicitly recognizing the cost attendant to e-discovery, the rule promotes cooperation by encouraging parties in commercial cases “to confer, at the outset of discovery and as needed throughout the discovery period, about [TAR] mechanisms they intend to use in document review and production.”  And so, the new Commercial Division Rule appears to bring New York State Commercial Division expectations closer in line with those set forth in the Federal Rules, specifically Rule 26(f), which encourages litigants (with an eye toward proportionality) to discuss preservation and production of ESI.

Questions about technology assisted review?  Please contact kcole@farrellfritz.com.

In a recent decision out of Oklahoma (Curtis v. Progressive N. Ins. Co., No. CIV-17-1076-C [W.D. Okla. June 13, 2018]), District Judge Robin J. Cauthron ruled that non-party ESI subpoenaed pursuant to Rule 45 was not subject to the 100 mile-limitation found in the Rule.  Specifically, the Court held there is “no violation of the 100-mile limitation,” as the non-party “subpoena at issue does not require the travel or attendance of any witnesses and Plaintiff is requesting the production of electronic documents.”

Factual Background

After plaintiff was involved in two automobile collisions, Curtis’ insurance company (“Progressive”) engaged Mitchell International, Inc. (“MII”), to create valuations of total loss for its use.  Eventually, Curtis filed the instant lawsuit against Progressive, claiming breach of contract, bad faith, unjust enrichment, and fraud in connection with the valuation of Curtis’ vehicles.  During discovery, Curtis served non-party MII with a subpoena duces tecum requesting the production of documents relating to “the correspondence, purchase, and analysis of the [computer valuation system]” MII used to create valuations of total loss for Progressive (“Subpoena”).  Curtis’ attorneys served the Subpoena upon MII using MII’s Oklahoma registered agent.  MII served written objections to the Subpoena, and meet and confer sessions failed to resolve the impasse reached between MII and Curtis.  And so, Curtis filed a Motion to Compel Compliance with Subpoena (the “Motion”).

MII argued the Court lacked jurisdiction to hear the Motion because MII’s headquarters and principal place of business are located in San Diego, and the Subpoena required compliance more than 100 miles away in Shawnee, Oklahoma.

In response, Curtis argued that her subpoena was valid and enforceable because “a subpoena that commands a person to travel beyond the 100-mile boundary must be quashed however, a Court retains discretion to command compliance with a subpoena for documents which requires production beyond the 100-mile limitation.”

In granting the Motion, Judge Cauthron noted that “Federal district courts enjoy broad discretion over discovery measures” and further stated:

“Here, Plaintiff states—and Mitchell does not dispute—that the information requested can be produced electronically. Mitchell has an Oklahoma registered agent and Progressive Northern Insurance Company continues to use the valuation system licensed and provided by Mitchell in Oklahoma to conduct business. As a result, Mitchell regularly transacts business in Oklahoma. The subpoena at issue does not require the travel or attendance of any witnesses and Plaintiff is requesting the production of electronic documents. This Court finds that there is no violation of the 100-mile limitation for electronic documents…”

This case is a good reminder that the Rule 45 geographic restrictions relates to how far a subpoena-recipient can be compelled to travel in order to comply with a subpoena (see FRCP 45[c]) and that while the geographic limitation applies equally to parties, and party officers, who cannot be commanded to appear for trial outside of the geographic restrictions set forth in the Rule (FRCP 45[c], [d][3][A][ii]), it has no applicability to the production of ESI.

The American Bar Association Ethics 20/20 Commission and Rule 1.1 provide that a lawyer’s duty of competence “[t]o maintain the requisite knowledge and skill, [requires] a lawyer [to] keep abreast of changes in law and its practice, including the benefits and risks associated with relevant technology.”  The New York County Lawyers’ Association Professional Ethics Committee Formal Opinion 749 (Feb. 21, 2017) echoes Rule 1.1 and discusses a lawyer’s “ethical duty of technological competence.”  Therefore, in today’s world replete with tweets, posts and handles, it is important that attorneys have some degree of social media savvy.

But what exactly does a lawyer need to know or do to be savvy when it comes to the internet of things and modern day technology?  For example, who knew there is a trend nationwide that lawyers have “a duty to Google.”  (See, e.g., Johnson v. McCullough 306 SW3d 551 [Mo. 2010] [the Court held that counsel had an affirmative duty to research jurors online]).  And, while New York has not gone so far as to require counsel to research jurors, New York does have a number of ethical opinions in place that govern how/when a lawyer may conduct such research should they choose to research a juror online.

The two opinions worth being aware of are the NY County Lawyer Association Opinion 743 (2011) (“NYCLA”) and the New York City Bar Association Commission on Professional Ethics Formal Opinion 2012-2 (2012) (“NYCBA”), which both provide that a lawyer may view the social media profile of a prospective juror provided that there is no communication with the juror.  The NYCLA Opinion further provides that a lawyer cannot seek to friend jurors, subscribe to their Twitter accounts or otherwise contact the juror.  Rather, a lawyer may only visit the prospective juror’s publicly available social media content.   In addition to lawyers’ conduct vis-à-vis prospective jurors, this same rule applies to sitting jurors.

The area rife for concern is what exactly is a “communication with a juror” under these Ethical Opinions?  And what happens if a lawyer intentionally or inadvertently communicates with a juror or prospective juror?

Use of social media by the tech-rube attorney could be rife with issues, as often times social media networks will generate an automatic notice (i.e., Katy Cole viewed your LinkedIn account).  Such notices are, under the ethical opinions, a violation of the no-communication rule.  It is therefore critically important that before you, a colleague or agent conduct any social media research about jurors, that you understand how the particular network operates.*

The ABA (Formal Rule 466 [April 2014]) reaches a different conclusion: that system generated notices are not communications. “The fact that a juror, or a potential juror may become aware that a lawyer is reviewing his internet presence when a network setting notifies juror or such, does not constitute a communication from the lawyer in violation of Rule 3.5(b).”   Below is a summary chart that may be a helpful tool when trying to determine whether you may view the social media profile of a prospective or sitting juror, the limitations of what you may/may not do and your attendant obligations, should you learn of a juror’s misconduct as a result of your viewing his/her social media content.

Pre-Trial Search:

NY County Lawyer Association

  • Lawyer may view social media profile of a prospective juror so long as there is no communication with the juror (whether initiated by the lawyer, her agent or automatically generated by the social media network) NYCLA, Formal Op. 743 (2011)
  • Lawyer cannot seek to “friend” jurors, subscribe to their Twitter accounts, send tweets to jurors or otherwise contact them. Id.
  • Lawyer may visit whatever is publicly available. Id.

NY City Bar Association

  • Lawyer may view the social media profile of a prospective juror so long as there is no communication with the juror (whether initiated by the lawyer, her agent or automatically generated by the social media network). NYCBA, Formal Op. 2012-2 (2012)

Mid-Trial Search:

NY County Lawyer Association

  • Lawyer may view the social media profile of a sitting juror so long as there is no communication with the juror (whether initiated by the lawyer, her agent or automatically generated by the social media networks). NYCLA, Formal Op. 743
  • Passive monitoring of jurors, such as viewing publicly available blog or Facebook page, may be permissible. Id.
  • Cannot make the juror aware of an attorney’s efforts to see the juror’s profiles on websites because might tend to influence the juror’s conduct in trial. Id.

NY City Bar Association

  • Lawyer may view the social media profile of a sitting juror so long as there is no communication with the juror (whether initiated by the lawyer, her agent or automatically generated by the social media network). NYCBA, Formal Op. 2012-2


NY County Lawyer Association

  • Even inadvertent contact with a prospective juror or sitting juror caused by an automatic notice generated by a social media network is technically a violation. NYCLA, Formal Op. 743
  • Viewing the public portion of a social media profile is ethical so long as there’s no message to the account owner of such viewing. Id.

NY City Bar Association

  • Even inadvertent contact with a prospective juror or sitting juror caused by an automatic notice generated by a social media network is technically a violation.  NYCBA, Formal Op. 2012-2
  • Viewing the public portion of a social media profile is ethical so long as there’s no message to the account owner of such viewing.** Such conduct is permissible even if juror might be unaware that information is publicly available, unless it is clear that juror intended the information to be private. Id.
  • A “friend” request or similar invitation or any other form that allows the juror to learn of the attorney’s viewing or attempted viewing is prohibited communication if the attorney “was aware that her actions would cause a juror to receive such a message or notification.” If attempts to research are inadvertent or unintended, then MAY be prohibited (makes note that mens rea is not a component). Id.

Reporting Obligations:

NY County Lawyer Association

  • In the event that a lawyer learns of a juror’s misconduct due to social media research, he/she must promptly bring it to the court’s attention. Id.
  • Cannot use knowledge of juror misconduct to their advantage. Id.

NY City Bar Association

  • In the event that a lawyer learns of a juror’s misconduct due to social media research, he/she must promptly bring it to the court’s attention. NYCBA, Formal Op. 2012-2
  • Attorney must use their best judgment in determining whether a juror has acted improperly. Id.
  • Cannot use knowledge of juror misconduct to their advantage. Id.


* Recall the “Hustle” mortgage fraud trial in the Southern District against Bank of America Corp. A juror complained that a first-year associate on the defense team had “cyberstalked him” on LinkedIn.   In that case, U.S. District Judge Jed S. Rakoff admonished defense attorneys after a juror sent him a note complaining “the defense was checking on me on social media.”  Although “It was a good faith misunderstanding,” according to the defense counsel, it was an embarrassing and potentially costly mistake.   It is therefore important to know which systems generate automatic notices of one’s viewing activity.

** If a lawyer logs into LinkedIn and clicks on a link to a Linkedin profile of a juror, an automatic message may be sent by LinkedIn to the juror whose profile was viewed advising of the identity of the LinkedIn subscriber who viewed the juror’s profile. In order for that reviewer’s profile not to be identified through LinkedIn, that person must change his or her settings so that he or she is anonymous or, alternatively, be fully logged out of his or her LinkedIn account.

Have questions on using social media for trial research?  Please contact me at kcole@farrellfritz.com.

In my December 2016 blog post, I wrote about how developing effective key words is very much an iterative and thought intensive process.  This message was recently reaffirmed by a decision out of the Southern District of Ohio, wherein the Judge reminded us that the process of identifying search terms it not merely one of guesswork.  Rather, it requires deliberative thought, testing and refining such that the searches yield target-rich and proportionate results.

In the complex commercial matter, AMERICAN MUNICIPAL POWER, INC., Plaintiff, vs. VOITH HYDRO, INC., Defendant, the parties were in litigation over the construction of four hydroelectric power plants (“Projects”) that spanned a decade.

When it came to discovery, the parties could not have been more oppositional in their respective stances.  Voith Hydro, Inc. (“Voith”) requested plaintiff conduct a search for ESI using only the four Project names (Cannelton, Smithland, Willow and Meldahl) as standalone search terms without qualifiers.

American Municipal Power, Inc., (“AMP”)  on the other hand, requested that Voith search its ESI collection without reference to any of the four project names; searching instead for various employee and contractor names connected in a boolean way with a list of common construction terms and the names of hydroelectric parts restricted only by date range.

However, both proposals were too expansive in their scope and, as a result, too burdensome.  For example, when proposing its search terms, defense counsel did not appreciate that not every shred of ESI that hit upon one of the four fairly common project names (i.e., Willow) would be relevant or, at the very least, worth examining for relevance.  As the Court noted, “As owner, AMP had retained millions of documents for more than a decade that contain the name of the four projects which refer to all kinds of matters unrelated to the case.”  The Project names –detached from any other qualifier or restriction – returned documents related to real property acquisitions, licensing, employee benefits, facility tours, parking lot signage, etc.  All content that “hit” on a search term, but none of which was relevant to the construction of the four hydroelectric power plants.  Thus, accepting Voith’s request would result in overly broad results not sufficiently tailored to the needs of the case.  And so, Judge Deavers ruled against the defense on the four project names keywords request, and granted a protective order for the plaintiff because, in the Judge’s words:

“The burden and expense of applying the search terms of each Project’s name without additional qualifiers outweighs the benefits of this discovery for Voith and is disproportionate to the needs of even this extremely complicated case.”

AMP’s more “expansive” search terms, however, were equally problematic.  Indeed, if the searches combined employee and contractor names together with a list of common construction terms and the names of hydroelectric parts, the results included innumerable confidential communications about other projects Voith performed for other customers.  Therefore, like Voith’s request, AMP’s search request similarly returned a large corpus of documents not relevant to the litigation.  AMP’s proposed searches also raised confidentiality concerns (i.e., commercial information about other customers).*

Keyword searching should be an iterative process that is tested and refined, and tested and refined again, before any commitments to search are made.

* AMP proposed to exclude the names of other customers’ project names with “AND NOT” exclusionary phrases.  This was deemed by the Court unworkable because Voith could not possibly reasonably identify all the projects from around the world with which its employees were involved during the decade they were engaged in work for AMP on the Projects.

Have questions on keyword searching?  Please contact me at kcole@farrellfritz.com. 

In 2016, Florida became the first state to mandate technology training for lawyers, when it adopted a rule requiring lawyers to complete three hours of CLE every three years “in approved technology programs.”  The requirement went into effect on January 1, 2017.  On April 20, 2018, the North Carolina State Bar Council approved a proposed amendment to the lawyer’s annual CLE requirement.  That amendment, if enacted, would mandate that one hour of the required twelve hours of annual CLE training be devoted to technology training (defined as a program, or a segment of a program, devoted to education on information technology (IT) or cybersecurity [see N.C. Gen. Stat. §143B-1320(a)(11)]).

While there is no indication that New York will be next to impose such requirements, it may only be a matter of time until other states (including New York) follow Florida and North Carolina’s lead.  Indeed, in a world where emails, tweets, texts and instant messages are a routine way of life and way of conducting business, lawyers should be expected to maintain a basic level of competence regarding technologies and electronically stored information.  Various Model Rules (see, e.g., ABA Model Rule 1.1, Comment 8)* and State Opinions (see, e.g., New York County Lawyers’ Association Professional Ethics Committee Formal Op. 749 [Feb. 21, 2017])** have already indicated that a lawyer’s duty of competence includes technological competence.  And, there is decisional law concluding that “professed technological incompetence is not an excuse for discovery misconduct.” James v. Nat’l Fin. LLC, No. CV 8931-VCL, 2014 WL 6845560 (Del. Ch. Dec. 5, 2014).

Because the electronic nature of today’s world is here to stay, it would make great sense to mandate regular training in areas of technology and cybersecurity.  Please contact me at kcole@farrellfritz.com if you are admitted to practice in New York and would like to be added to my technology CLE invitee list.


*American Bar Association Model Rule 1.1  – “Duty of Competence.”  Comment 8 : “. . . a lawyer should keep abreast of changes in the law and its practice, including the benefits and risks associated with relevant technology . . .”

** New York County Lawyers’ Association Professional Ethics Committee Formal Op. 749 (Feb. 21, 2017) discussing the “ethical duty of technological competence with respect to the duty to protect a client’s confidential information from cybersecurity risk and handling e-discovery” in a litigation or government investigation.

When tasked with a document review project, there are various analytic tools available to streamline the process in order to improve efficiency and accuracy.  We’ve already discussed certain of these tools (see April 26 post discussing predictive coding and May 16 post discussing email threading).  Today’s post focuses on another, interrelated tool: document clustering.

What is Document Clustering?

As you can imagine, the way in which a cache of documents is organized for review can make a tremendous difference in not only the efficiency of the review, but also the accuracy of the review itself.  Clustering software examines the text in documents, determines which documents are related to each other, and groups them into clusters.  Clustering performs the electronic equivalent of putting your documents into labeled boxes so that things only end up in the same box if they belong together. Clustering groups similar documents together and then assigns those document to the same reviewer(s), allowing for a more efficient review as related documents can be reviewed together.  Clustering organizes the documents according to the structure that arises naturally, without query terms.  It labels each cluster with a set of keywords, providing a quick overview of the cluster; basically telling you, the project lead, what the documents have in common at a conceptual level. The keywords give a quick idea of what each cluster is about, allowing you to easily identify the themes of your document set.  For example, if you are a litigator looking for information about a particular contract and the cluster is about the Company’s summer softball team, documents in that cluster are not relevant.  During review, you can, with a single mouse click, categorize or tag a single document, a cluster of documents, or a set of clusters containing a specific combination of keywords. *


*Certain clustering software has an automatic categorization capability, where all documents sufficiently similar to a set of documents can be categorized the same way, greatly reducing the amount of labor needed when new documents are added to a case.  It enables you to leverage the labor you’ve put into categorizing the earlier documents.

The April 26 blog post discussed predictive coding as one of many analytical tools available to empower attorneys to work smarter, thereby reducing discovery costs and allowing attorneys to focus sooner on the data most relevant to the litigation. Another tool in the litigator’s arsenal that can promote efficiency during document review is email threading.

According to The Radicati Group, the average employee sends 36 emails per day. That extrapolates to approximately 10,000 emails per employee per year.  And so, even in cases involving a limited number of custodians, the volume of emails at issue can be significant.

Email threading provides a number of benefits, including eliminating the need to review the same content multiple times and minimizing potential for inconsistent coding of emails.

So what exactly is threading?  It is a process by which emails are grouped together so they can be reviewed as a single coherent conversation.  For example, if I write John Smith an email, John’s reply will very likely include my original message at the bottom of the email chain.

When an email collection for discovery purposes occurs, both segments of the email chain are collected.  Presuming my conversation with John continues, the email may have many segments over the course of days /weeks.

In email threading, an algorithm compares and matches segments, resulting in emails from the same conversation being grouped together.  Then, the most inclusive email (i.e., the one with the most complete content) is promoted for review by the review team. Non-inclusive emails (i.e., those with text and attachments contained in another inclusive email) are suppressed.  By reviewing inclusive messages, rather than non-inclusive messages, the review team bypasses redundant content and limits the number of documents to review.

Threading also allows the reviewer to see the full picture.  For example, if an email conversation has 16 segments, and those segments are spread among various reviewers, they would appear as separate messages with no particular order, allowing for the first segment to be reviewed 16 times (once on its own, then as a segment in the second message, and in the third message, and so on) by multiple different reviewers.   And, if the first three segments are non-responsive, what happens when the fourth segment is responsive?  Do you have to search for the first three segments?

In summary, email threading should be implemented in every case.  It makes review more efficient, more consistent, and can streamline even the smallest review project.

Traditional document review can be one of the most variable and expensive aspects of the discovery process.  The good news is that there are innumerable analytic tools available to empower attorneys to work smarter, whereby reducing discovery costs and allowing attorneys to focus sooner on the data most relevant to the litigation.   And, while various vendors have “proprietary” tools with catchy names, the tools available all seek to achieve the same results:  smarter, more cost effective review in a way that is defensible and strategic.

Today’s blog post discusses one of those various tools – predictive coding.  The next few blog posts will focus on other tools such as email threading, clustering, conceptual analytics, and keyword expansion.

Predictive Coding

Predictive coding is a machine learning process that uses software to take keyword searches / logic, entered by people, for the purpose of finding responsive documents, and applies it to much larger datasets to reduce the number of irrelevant and non-responsive documents that need to be reviewed manually.  While each predictive algorithm will vary in its actual methodology, the process at a very simplistic level involves the following steps:

  1.  Data most likely relevant to the litigation is collected. Traditional filtering and de-duplication is applied.  Then, human reviewers will identify a representative cross-section of documents, known as a “seed set,” from the remaining (de-duplicated) population of documents that need to be reviewed.   The number of documents in that seed set will vary, but it should be sufficiently representative of the overall document population.
  2.  Attorneys most familiar with the substantive aspects of the litigation code each document in the seed set responsive or non-responsive as appropriate. Mind you, many of the predictive coding software available allows users to perform classification for multiple issues simultaneously (i.e., responsiveness and confidentiality).  These coding results will then be input into the predictive coding software.
  3.  The predictive coding software analyzes the seed set and creates an internal algorithm to predict the responsiveness of other documents in the broader population.  It is critically important after this step that the review team who coded the seed set spend time sampling the results of the algorithm on additional documents and refine the algorithm by continually coding and inputting sample documents until desired results are achieved.  This “active learning” is important to achieve optimal results.  Simply stated, active learning is an iterative process whereby the seed set is repeatedly augmented by additional documents chosen by the algorithm and manually coded by a human reviewer. (This differs from “passive learning,” which is an iterative process that uses totally random document samples to train the machine until optimal results are achieved).

Once the team is comfortable with the results being returned, the software applies the refined algorithm to the entire review set and codes all remaining documents as responsive or unresponsive.





When faced with the task of collecting, processing, reviewing and producing digital data, law firms (and clients) often retain outside vendors to assist.  Depending on the vendor, and the circumstances of the retention, there may be a single vendor retained to handle the entire spectrum of client needs (i.e., from collection to production).  Or, there may be a series of vendors retained (i.e., one to perform a forensic collection, another to handle document review).   Before retaining any vendor(s), however, it is advisable to perform some minimal due diligence on the vendor in an effort to minimize the potential that client data could be compromised.    Indeed, in today’s age of digital data and increased efforts to ensure data privacy and protection, it is critically important that any vendor that will have access to a client’s data be obligated to keep the data in an environment equally as secure as the environment in which the organization and/or law firm maintained the data.

Below is a suggested list of questions /topics to discuss with vendors before retaining them.  The list is by no means exhaustive.

  • Does the vendor have an incident response plan?
  • Does the vendor have any security certifications?  For example, the International Standards Organization (“ISO”) 27001 — the international standard for information security.
  • Does the vendor have cyber liability insurance? If so, is the insurance adequate?
  • Will the vendor permit security audits or provide a copy of the most recent security audit report?
  • Has the vendor suffered data security breaches/events?
  • What are the vendor’s encryption practices?  And, do these practices apply only to data it houses, or also to data in transit?

It is also advisable to include in the vendor agreement that the vendor must notify you/client of any data incidents within a set time frame.

When one preserves and collects electronic data for a litigation, one typically casts a broad net.  This, in turn, can result in the preservation and collection of a significant volume of documents that are not relevant to the dispute at hand.  In an effort to identify the most likely relevant documents from the cache that has been broadly preserved and collected, lawyers tend to use search terms and keywords.  But, as anyone who has engaged in that process knows, due to the range of language used in everyday communications, even the most targeted search terms yield results that are not relevant (i.e., “false hits”). So how can a practitioner best gauge the overall effectiveness of their document collection and review process?

Enter PRECISION and RECALL — the two metrics that best assess effectiveness.

So what exactly is precision and recall?


Precision measures how many of the documents retrieved are actually relevant.  For example, a 75 percent precision rate means that 75 percent of the documents retrieved are relevant, while 25 percent of those documents have been misidentified as relevant.


Recall measures how many of the relevant documents in a collection have actually been found. For example, a 60 percent recall rate means that 60 percent of all relevant documents in a collection have been found, and 40 percent have been overlooked.

It is relatively easy to achieve high recall with low precision if you collect robustly.  The downside is you will also retrieve a lot of irrelevant information, which in turn will increase the cost of review.   Similarly, high precision with low recall is easy to achieve.  By keeping your key word searches few and narrow, you will likely retrieve mostly relevant documents; and review costs will be contained because you will collect only relevant information. Many relevant documents, however, will also be overlooked.

The ideal result is to achieve high recall with high precision.  But identifying only the necessary information and little else is a task difficult to achieve.  In order to maximize your chance of achieving high recall with high precision, consider using a combination of temporal limitations, search terms that are vetted with the individuals most familiar with the intricacies of the case and its underlying facts, and early analytics to assess the validity of the terms chosen.