Kathryn Cole

Subscribe to Kathryn Cole's Posts

When faced with the task of collecting, processing, reviewing and producing digital data, law firms (and clients) often retain outside vendors to assist.  Depending on the vendor, and the circumstances of the retention, there may be a single vendor retained to handle the entire spectrum of client needs (i.e., from collection to production).  Or, there may be a series of vendors retained (i.e., one to perform a forensic collection, another to handle document review).   Before retaining any vendor(s), however, it is advisable to perform some minimal due diligence on the vendor in an effort to minimize the potential that client data could be compromised.    Indeed, in today’s age of digital data and increased efforts to ensure data privacy and protection, it is critically important that any vendor that will have access to a client’s data be obligated to keep the data in an environment equally as secure as the environment in which the organization and/or law firm maintained the data.

Below is a suggested list of questions /topics to discuss with vendors before retaining them.  The list is by no means exhaustive.

  • Does the vendor have an incident response plan?
  • Does the vendor have any security certifications?  For example, the International Standards Organization (“ISO”) 27001 — the international standard for information security.
  • Does the vendor have cyber liability insurance? If so, is the insurance adequate?
  • Will the vendor permit security audits or provide a copy of the most recent security audit report?
  • Has the vendor suffered data security breaches/events?
  • What are the vendor’s encryption practices?  And, do these practices apply only to data it houses, or also to data in transit?

It is also advisable to include in the vendor agreement that the vendor must notify you/client of any data incidents within a set time frame.

When one preserves and collects electronic data for a litigation, one typically casts a broad net.  This, in turn, can result in the preservation and collection of a significant volume of documents that are not relevant to the dispute at hand.  In an effort to identify the most likely relevant documents from the cache that has been broadly preserved and collected, lawyers tend to use search terms and keywords.  But, as anyone who has engaged in that process knows, due to the range of language used in everyday communications, even the most targeted search terms yield results that are not relevant (i.e., “false hits”). So how can a practitioner best gauge the overall effectiveness of their document collection and review process?

Enter PRECISION and RECALL — the two metrics that best assess effectiveness.

So what exactly is precision and recall?

Precision

Precision measures how many of the documents retrieved are actually relevant.  For example, a 75 percent precision rate means that 75 percent of the documents retrieved are relevant, while 25 percent of those documents have been misidentified as relevant.

Recall

Recall measures how many of the relevant documents in a collection have actually been found. For example, a 60 percent recall rate means that 60 percent of all relevant documents in a collection have been found, and 40 percent have been overlooked.

It is relatively easy to achieve high recall with low precision if you collect robustly.  The downside is you will also retrieve a lot of irrelevant information, which in turn will increase the cost of review.   Similarly, high precision with low recall is easy to achieve.  By keeping your key word searches few and narrow, you will likely retrieve mostly relevant documents; and review costs will be contained because you will collect only relevant information. Many relevant documents, however, will also be overlooked.

The ideal result is to achieve high recall with high precision.  But identifying only the necessary information and little else is a task difficult to achieve.  In order to maximize your chance of achieving high recall with high precision, consider using a combination of temporal limitations, search terms that are vetted with the individuals most familiar with the intricacies of the case and its underlying facts, and early analytics to assess the validity of the terms chosen.

 

In 2012, Klipsch Group Inc. (“Klipsch”), a manufacturer of sound equipment, filed a complaint against ePRO E-Commerce Ltd. (“ePRO”), alleging an ePRO subsidiary was selling counterfeit headphones.  Through discovery demands, Klipsch called for the production of information relevant to the sale of the allegedly infringing product, including emails and specific sales data.    Eventually, however, it became clear that ePRO was not engaging in a cooperative discovery process but instead was avoiding its discovery obligations.  For example, ePRO:  failed to implement an appropriate legal hold notice even after having been directed by the Trial Court to do so; limited vendor access to electronic data; failed to produce many responsive documents; and (as demonstrated by a forensic examination authorized by the Court) engaged in routine and systematic deletion of thousands of files and emails using a data wiping software long after the suit had commenced.

Because of the numerous and continuous discovery failures, Klipsch moved for sanctions and ultimately filed an ex parte motion seeking additional relief.  The District Court concluded that ePRO willfully spoliated evidence and it imposed various sanctions on ePRO including:

(1)   a jury instruction requiring the jury find that ePRO destroyed relevant emails and related data;

(2)  a jury instruction permitting the jury to infer that the destroyed evidence would have been favorable to Klipsch; and

(3)  Klipsch’s reasonable costs and fees, which the Court ultimately concluded was $2.7 million necessitated by ePRO’s obstructionist behavior.

ePRO filed an interlocutory appeal, arguing that the District Court’s $2.7 million sanction in the case where damages were, at most, $20,000 was impermissibly punitive and grossly disproportionate.

In January, the Second Circuit upheld the District Court’s sanction.  In doing so, the Circuit held that discovery sanctions should be commensurate with the costs occasioned by the sanctionable behavior, not the value attributable to the alleged (or even proven) compensatory damages.  To allow otherwise would, according to the Circuit, force a litigant to a small value dispute to beat risk to suffer blatant and egregious discovery misconduct.  And so, sanctions must be proportionate to the costs inflicted on a party – irrespective of total case value – by virtue of that party having to remediate discovery misconduct by its adversary.

Consistent with the theme of cooperative discovery, the Second Circuit noted that “the integrity of our civil litigation process requires that the parties….carry out their duties to maintain and disclose the relevant information in their possession in good faith.”    Like the countless other cases I have blogged about since December 2015, this decision serves as another reminder that judges expect cooperation between the parties and their attorneys during the litigation process to achieve orderly and cost-effective discovery; indeed, it is a priority.  Had ePRO and its counsel simply cooperated with their adversary and engaged in good faith discovery, the outcome here would have been entirely different.*

 

 

* Cooperation among counsel is critically important and the means to insure compliance with Rule 1’s mandate that the parties are responsible for securing the “just, speedy and inexpensive determination” of a civil litigation.  Indeed, the revised committee notes state, “[m]ost lawyers and parties to cooperate to achieve these ends” and “[e]ffective advocacy is consistent with – and indeed depends upon – cooperative and proportional use of procedure.”

Imagine if the above emojis, casually fired off in a text message (or in an Instagram or Facebook post) to a friend or colleague, could be used against you as evidence of workplace harassment?

Or if another combination of cartoon-like representations of emotions could be used as proof of defamation?

Or if inclusion of a face emoji with its tongue sticking out could preclude a reasonable reader from concluding the potentially defamatory statement was anything other than a joke?

Some disbelieving readers may think, never!  But, not so fast.  In fact, there are a growing number of cases, both in the United States and elsewhere, where emoji images have been entered as evidence requiring the judge or the jury (as the case may be) to interpret what exactly was meant by the emoticon.  But therein lies the issue —  what exactly does the combination of emojis (or a small digital image or icon used to express an idea, or emotion) in any given text or communication mean?  There is, after all, no fixed emotional resonance or clear dictionary definition for interpreting them.  So, while one colleague may interpret the smirking face with a beer mug as nothing more than an innocent invitation to grab a drink, another colleague may interpret that same message to have a potentially sinister motive.   It is this very subjective nature of emojis, and the double-meaning of some emojis, that can cause issues in the workplace and elsewhere.

So what is a business to do?  Whether you   or   emojis, it is important to recognize they are here to stay and part of mainstream communication.  Regulating their use in the same way that other communications are regulated may be an advisable business practice.  For example, consider whether there should be rules governing emojis in office communications.  If so, review and update your employee handbook.    

In Youngevity Intl’s Corp. v. Smith (No: 16-cv-00704 [SD CA December 21, 2017]), defendants sought an Order pursuant to Federal Rules of Civil Procedure 26(g) and 37.  The Order required Plaintiffs to remediate an improper discovery production to pay for Defendants’ costs for bringing the motion to compel and for the cost to review various improper prior productions.  Specifically, in connection with the discovery of electronically stored information (“ESI”), Defendants proposed a three-step process by which: “(i) each side proposes a list of search terms for their own documents; (ii) each side offers any supplemental terms to be added to the other side’s proposed list; and (iii) each side may review the total number of results generated by each term in the supplemented lists (i.e., a ‘hit list’ from our third-party vendors) and request that the other side omit any terms appearing to generate a disproportionate number of results.”

Approximately one week later, Plaintiffs advised in writing that they were “amenable to the three step process described in your May 9 e-mail.”  The parties then exchanged lists of proposed search terms to be run through their own ESI and the ESI of their opponent.

Pursuant to the agreed-to three-step process, Defendants provided to Plaintiffs its “hit list.”  Plaintiffs, however, never produced its “hit list.”  Instead, Plaintiff produced two large caches of documents – the first consisting of approximately 1.9 million pages and the second production consisting of approximately 2.3 million pages.   Upon receipt by Defendants, it became clear that the productions had been bulk coded with a CONFIDENTIAL legend and in some instances also with an ATTORNEYS’ EYES ONLY designation.  The produced materials also contained non-responsive documents.  A few months later, defendants advised they inadvertently failed to produce an additional 700,000 documents due to a vendor error.  Although the parties attempted to resolve amicably their differences, they were unsuccessful.

As a result, Defendants’ filed the instant motion to compel proper production and for costs.

In granting Defendants’ motion, Magistrate Judge Jill L. Burkhardt concluded, “the record indicates that Youngevity did not produce documents following the protocol to which the parties agreed.”  Specifically, “Youngevity failed to produce its hit list … and instead produced every document that hit upon any proposed search term” thus conflating “a hit on the parties’ proposed search terms with responsiveness.”  Moreover, the Court observed “the parties negotiated a stipulated protective order, which provides that only the ‘most sensitive’ information should be designated as AEO.”  As a result, Judge Burkhardt gave the plaintiffs two options for correcting their discovery productions with specific deadlines:

“1) By December 26, 2017, provide its hit list to Defendant; by January 5, 2018, conclude the meet and confer process as to mutually acceptable search terms based upon the hit list results; by January 12, 2018, run the agreed upon search terms across Plaintiff’s data; by February 15, 2018, screen the resulting documents for responsiveness and privilege; and by February 16, 2018, produce responsive, non-privileged documents with only appropriate designations of “confidential” and “AEO” (said production to include that subset of the not-previously-produced 700,000 documents that are responsive and non-privileged); or

2) By December 26, 2017, provide the not-previously-produced 700,000 documents to Defendant without further review; pay the reasonable costs for Defendant to conduct a TAR of the 700,000 documents and the July 21, 2017 and August 22, 2017 productions for responsiveness; by January 24, 2018, designate only those qualifying documents as “confidential” or “AEO”; by that date, any documents not designated in compliance with this Order will be deemed de-designated.”

Judge Burkhardt also ordered Plaintiffs to pay for the reasonable expenses, including attorney’s fees, for bringing the motion and for the expenses incurred by Defendants “as a result of Youngevity’s failure to abide by the Stipulated Protective Order.”

Conclusion

This case is another reminder of what appears to be the well-embraced theme in Federal discovery – cooperation.  The 2015 amendments made plain that cooperation between the parties and their attorneys during the litigation process to achieve orderly and cost-effective discovery is a priority.  Indeed, mutual knowledge of the relevant facts is essential to proper litigation; and therefore the process of obtaining those facts (i.e., discovery) should be a cooperative one.  Had counsel simply abided by the three-step process and stipulated protective Order it willingly entered, there would be no need to defend against (and foot the bill for) the motion to compel.

You are involved in litigation and faced with a document review need, what now? Naturally you need to find attorneys to review these documents. To this end, depending on the volume of data at issue, many firms will either: (1) staff the document review with firm attorneys, or (2) work with a vendor to retain a review team comprised of contract attorneys. Irrespective of who conducts the needed review, the cost attendant to that review and the time to complete the review is often a concern.  Because a party to a litigation should not produce documents without reviewing them, predictive coding may be a particularly helpful option.

Simply put, predictive coding is the use of a computer system to help determine which documents are relevant to a particular legal proceeding.  The system makes this determination based upon “training” it receives from human input.  In fact, for a predictive coding system to make accurate decisions, the system needs direction from humans fluent in the intricacies of the lawsuit.  During this training phase, attorneys will review a seed set of documents and code those documents accordingly (i.e., responsive, privilege, tagging issues applicable). (FN*) At each step of this process, the computer system is being trained and educated. Refinements are made along the way and internalized by the system. Once trained, the computer will find and code (based on its training) the responsive documents far quicker (and often with far greater accuracy) than human reviewers. Specifically, the computer will build a model to identify documents that have a high probability of correct classification into categories pre-defined through the training /seed coding.

As with any review (entirely human or a combination of human and machine review), a validation process should be implemented.  Specifically, there should be a work flow created that provides for attorney reviewers to check the efficacy and accuracy of the model.   It is important here to determine what validation/QC process is best implemented.  For example, one can implement a statistical sampling of data where documents are selected at random and reviewed for accuracy.  This sort of validation would be reflective of the machine’s overall accuracy and reflective of the overall document population.  There is also, however, a more particularized sampling where a group of relevant documents are selected from the population and reviewed for accuracy.  This sort of validation would be more limited in that it would not allow the attorney running the review to form any conclusions about the entire document population.  (FN**)

Because of the ever-increasing volume of data and information, predictive coding is becoming a more attractive tool to incorporate into every document review to some degree, especially because no minimum data size is required to use predictive coding.  A document review that uses predictive coding coupled with a well-devised work flow will inevitably minimize review costs while maximizing efficiency during the review.

 

FN* Because the coding on these seed documents will impact the quality of the computer’s determinations, it is important the individuals coding the seed documents understand well the lawsuit and how the predicting coding system is to work.

FN**  And, if you are not comfortable allowing a computer to do that much work, other predictive coding options (e.g., other than allowing the system to extrapolate based upon seed sets) are available.  For example, prioritized review can be used whereby the system identifies and escalates important documents for review but keeps likely irrelevant documents in the queue.  Incorporating this option into your work flow allows attorneys to still lay eyes on all documents but provides for an efficient prioritization of documents that must be reviewed.

What do applications like Snapchat, Telegram, Wickr, Cover Me, Speak On, and Whisper have in common? They are all self-destructing message (“SDM”) applications. What exactly does this mean, you ask? Self-destructing messaging applications transmit information with end-to-end encryption, and auto destruct after a set time period of time, or after receipt and access by the intended recipient.  Consider Snapchat, for example. Snapchat is one of the most popular social media platforms in the world. Indeed, in 2016, Snapchat surpassed Facebook’s number of video views per day.  Part of Snapchat’s popularity is derived from the fact that the user can set timers for shared photos / videos to self-destruct once the person received it; allowing users (typically younger generations) to share photos without the risk of the photo going public.

Yet, what happens when SDM technologies (which are evolving rapidly) are used in the corporate world?  How does one preserve potentially relevant information?  What is the risks verses benefits of incorporating into one’s business SDM technology?  These questions – and others – are likely questions litigators will grapple with in the coming months/years given the rapid growth of SDM technology.**

While it is impossible to predict the future, I suspect it is only a matter of time until this issue becomes more of a focus in litigation and I look forward to reading decisions on point as the case law catches up to the technology.

** Consider, for example, the Waymo LLC v. Uber Technologies, Inc., lawsuit, wherein allegations have arisen that one party is hiding information relevant to the lawsuit by transmitting that information via SDM. Consider further the fact that the Department of Justice in December issued an enforcement policy urging strongly against the use of messaging applications that do not store data in a way that allows for access during a subsequent investigation.  These recent lawsuits and policies make plain SDM technology is being employed in corporate America.

 

 

In IDC Financial Publishing, Inc. v. Bonddesk Group, LLC (15-cv-1085-pp, 2017 WL 4863202 (E.D. Wis. Oct. 26, 2017)), the Eastern District of Wisconsin granted IDC’s motion to compel the production of more than 600 documents that had previously been produced by Bonddesk with extensive non-responsive redactions applied.

Bonddesk argued that the applied redactions were necessary to protect confidential business information that had no relevance to the underlying dispute.  In making this argument, Bonddesk relied on In re Takata Airbag Prods. Liab. Litig., 2016 WL 1460143 (S.D. Fla. Feb. 24, 2016). In the Takata case, the district court in the Southern District of Florida permitted the redactions based upon “non-responsiveness,” because of the “concern that the documents contained competitively sensitive materials that may have been exposed to the public, despite protective orders.”

In the present case, however, the Court looked to the Federal Rules for guidance. Specifically, the Court observed that redactions based upon relevance are not explicitly supported by the Federal Rules, and seemingly contrary to the Rules’ allowance for broad discovery. The Court also reasoned redactions based upon relevance (or a lack thereof), have the potential for abuse. For example, should non-responsive redactions be deemed per se proper, parties may be incentivized to “hide as much as they dare” (internal citations omitted).   Moreover, the Court reasoned that Bonddesk failed to provide an otherwise “compelling reason” for its “extensive” redactions and failed to explain why the protective order in place did not provide adequate protection.

Therefore, based upon the Court’s fear of party abuse, Bonddesk’s failure to articulate a compelling reason for the redactions, and Bonddesk’s failure to articulate why the protective order was insufficient, the Court granted IDC’s motion to compel. The Court concluded it “[did] not see a compelling reason to alter the traditionally broad discovery allowed by the rules by letting the defendants unilaterally redact large portions of their responsive documents on relevance grounds.”

This decision does not conclude relevance redactions are per se impermissible, but it reminds us, as practitioners, that engaging our adversary in a meaningful discussion about inter alia redactions, is good practice. Recall, the theme of late from the federal courts is “cooperation”. Here, there was a protective order in place and no compelling reason to apply the redactions at issue. Query whether had the lawyers spoken, the cost of motion practice and a subsequent (un-redacted) production could have been avoided. It seems lately that a collaborative approach is preferred by the Bench.  However, if such a meet and confer/discussion is not practical or possible, at least be prepared to provide compelling reasons why a robust “So Ordered” confidentiality agreement is not sufficient to protect the content of your client’s documents before unilaterally applying redactions.

Earlier this year, I wrote about the then-proposed changes to the Federal Rules, and how those changes (if implemented), could impact electronic discovery. (February 15, 2017 blog)  Well, the time has come — effective December 1, 2017, the amendments to Federal Rule of Evidence 902 “Evidence That is Self Authenticating” went live.

As the title suggests, Federal Rule of Evidence (“FRE”) 902 applies to evidence that is self-authenticating (i.e., sealed and signed public documents, certified copies of public records, newspapers).  Because such documents are deemed “self-authenticating,” attorneys do not need to go through the authentication process in court with qualified expert testimony.  Effective December 1st, two new categories of documents will qualify as self-authenticating, too.

Specifically, 902(13) and (14) are the newly added provisions – each of which apply to electronically stored documents.

Subsection 13 provides:

(13) Certified Records Generated by an Electronic Process or System. A record generated by an electronic process or system that produces an accurate result, as shown by a certification of a qualified person that complies with the certification requirements of Rule 902(11) or (12). The proponent must also meet the notice requirements of Rule 902(11).

And, subsection (14) provides:

(14) Certified Data Copied from an Electronic Device, Storage Medium, or File. Data copied from an electronic device, storage medium, or file, if authenticated by a process of digital FEDERAL RULES OF EVIDENCE 3 identification, as shown by a certification of a qualified person that complies with the certification requirements of Rule 902(11) or (12). The proponent also must meet the notice requirements of Rule 902(11).

Subsection (13) applies to machine-generated information (i.e., produced by a computer system or computer process) and is analogous to Rule 902(11)’s certification of business records.  Subsection (14) applies more broadly to copied/replicated ESI provided the copy retains a hash value that is identical to the original.[1] Subsection 14, thus, effectively dispenses with the costly need for trial testimony of a forensic or technical expert where best practices are employed, as certified through a written affidavit by a “qualified person.”

While neither subsection (13) nor (14) dispense with the need to demonstrate authenticity, the new provisions drastically simplify the process.  Indeed, the expectation is that the new Rules will provide a streamlined and efficient process to establish a foundation for ESI collected in a Rule 902(14) compliant manner. This will increase predictability by eliminating surprise challenges, and will encourage the use of ESI practitioners by allowing written certifications in the place of expensive and time-intensive in-person testimony.  Indeed, the ability to eliminate foundational testimony will undeniably result in significant cost savings to one’s client and help promote judicial efficiency.[2]

[1] Recall, a file’s hash value is often likened to its fingerprint – a unique identifier attributable to the contents of a file being processed through a cryptographic algorithm, which results in a unique numerical value – the hash value – being produced that identifies the contents of the file.

[2]  However, this necessarily presupposes that practitioners in the federal courts will understand what a 902(14) compliant collection means.

In Barcroft Media, Ltd. et al. v. Coed Media Grp., LLC, No. 16-CV-7634 (JMF) (S.D.N.Y. Sept. 28, 2017), Plaintiffs – providers of entertainment-related photojournalism and owners of celebrity photographs – interposed various intellectual property claims against Defendant Coed Media Group, LLC (“CMG”).  The claims related to the allegedly infringing use of certain celebrity photographs (the “Images”) on CMG’s pop culture and celebrity gossip websites.  Because CMG purportedly failed to preserve the “webpages” on which it displayed the Images, Plaintiffs filed a motion for spoliation sanctions pursuant to Federal Rule of Civil Procedure 37.

In deciding the sanctions motion, Judge Furman discussed the relevant provisions of Rule 37 and its 2015 amendment.  Specifically, the Court noted that a sanction may be imposed only if the ESI that should have been preserved is lost because a party failed to take reasonable steps to preserve it and the ESI cannot be restored or replaced through additional discovery.  Once that standard is met, the next step in the inquiry is to determine whether; (1) the non-offending party has been prejudiced from the loss of ESI; and (2) the offending party acted with the intent to deprive another party of the information’s use in the litigation. Even a cursory reading of the (not so newly) amended Rule 37 makes plain that mere loss of data alone is not enough for sanctions. Rather, loss coupled with a prejudice is necessary and, even then, the resulting sanction must only be as great as needed to cure the prejudice. Thus, only after a Court identifies a prejudice to the aggrieved party, may the Court order measures necessary to remediate that prejudice.

Against this backdrop, the Court concluded Plaintiff’s motion for sanctions was without merit and bordered on frivolous.  Specifically, the Court found there was no foundation for the imposition of any sanctions.

“Given the plain language of [Federal Rule 37(e)], Plaintiffs’ motion borders on frivolous, for the simple reason that they cannot even show that the evidence at issue was ‘lost.’ Several of the images are still hosted on CMG’s websites…And the record makes clear that Plaintiffs themselves possess copies of the other Webpages—in the form of screen captures taken when they displayed the Images (the ‘Screenshots’)…In fact, Plaintiffs themselves list the Screenshots as trial exhibits…Given that…, there is no foundation to impose sanctions under Rule 37(e).”

The Southern District went on to conclude that “Plaintiffs obviously cannot show prejudice ‘as [they] actually possess[ ] copies’ of the relevant evidence” and sanctions are not appropriate.[1]

This decision serves as an important reminder that practitioners need to remain current in their understanding of the Federal Rules and the standards articulated under those Rules.  Indeed, a sanction for lost ESI cannot be predicated merely upon loss alone.  Rather there must be a loss of relevant ESI coupled with a prejudice before sanctions may be imposed.

[1] Bear in mind the decision is limited to spoliation issues, not authenticity and best evidence.