An Interview with Mark Joing
Founder & CEO of Mojo Trials
Strategikon: What, from your perspective, is the most painful part of outsourcing for our industry?
MJ: In my 20 plus years’ experience, the most painful aspect of outsourcing is the lack of information. This lack of information can lead to choosing the wrong vendors for a study and to suboptimal outcomes in trials. As a small Sponsor, we try to select the vendors that we think will be the best fit for our company and for the trial. But I’d estimate that the percentage of time we actually make a great choice is maybe 50%. The gap isn’t due to lack of effort or specificity – we actually create very complex RFPs, get very detailed responses, and do bid defenses with a number of vendors before choosing. Despite all this effort, we just don’t have the performance insights to help us judge whether a vendor is really a good fit for us. Let me add that once you select a vendor for a study or a program, it’s extremely hard to change. That makes it even more important to make the right decision in the first place.
Strategikon: So, changing vendors is almost like starting over?
MJ: Exactly. I’ve had studies where I’ve had to change a central lab or similar vendor. Everything in our process has been set up based on the system we first chose, so all your sites are using that vendor. It’s extremely painful to switch. Picking the right vendors and making that ‘right call’ initially is so very important.
Big companies may have a better average because they have direct vendor experience through other trials, but as a small Sponsor, we usually have to base our assessments on other companies’ experience with a vendor; at previous companies, from industry colleagues’ experiences, or by calling references. But often that doesn’t work. One example: we selected a central lab for one of the studies I worked on. We hadn’t worked with this lab before, but we did a very careful and thorough assessment of all the candidates. Five or six different labs bid on the study – which was a lot. We did reference checks. We interviewed the key team members. We did a vendor audit. And we thought we had selected a vendor that would be a great fit for study. That was definitely not the case. Our team ended up spending half of our time dealing with just this vendor. It started with very basic things like not sizing the blood tubes correctly and escalated to freezing the production of lab kits for several weeks because they had an unrelated internal issue. Had we known how this vendor would perform we would have never selected them. We discussed switching but ended up sticking with the vendor because of all the additional problems that replacing them would create. Ultimately, we got the results we needed from the study, but it was way more work than we planned and could have been prevented by selecting the right vendor at the outset.
“When you can run other scenarios, you can go back to senior management and say, “If we did this as an eight-week study it could be an $8 million study instead of $10 million study.”
Mark Joing
Strategikon: For clarification, what kind of information would be most useful to make better outsourcing choices?
MJ: Great question. I’d love to have access to unbiased, objective, factual vendor performance information. Vendors, of course, are going to promote their successes. The problem is that Sponsors have very limited access to information about actual vendor performance or cost unless we’ve already worked with them. What we need is a “Yelp” for clinical vendors. I would love to get information about how many Sponsors go back to the same vendor again for the next study opportunity: of the last 100 studies that you supported as a vendor, how many of those customers came back and chose you again? Or that this vendor has had huge turnover in their customer base and is getting little repeat business. That’s the type of information that I think is lacking, especially for small Sponsors, but these vendors don’t want that information out there. That central lab in my example doesn’t want any other Sponsors to know the kind of problems we had with them. Clearly, they will never use us as a reference, so the only way that another Sponsor is to get this insight is if they happen to know me personally.
Clinical Maestro isn’t “Yelp”, but it provides at least some assistance with this problem. Not with vendor ratings but using a system like it allows for greater control of the proposal process and a better apples-to-apples comparison, including more control over what questions are asked and better alignment of the responses. It doesn’t directly address the pain point, but it increases confidence since the vendors provide the information requested in an easily digestible format. That’s a good part of the equation.
“If we had a tool like Clinical Maestro’s LEAD module for accruals and forecasting using vendor-reported actuals, that [the extra expense] could have been avoided because we would have had more insight before entering this arrangement and we would have seen the trajectory of variance to budget sooner instead of getting surprised.”
Mark Joing
Strategikon: What is the biggest challenge facing those responsible for clinical trial operations?
MJ: As a small Sponsor, we’re extremely reliant on vendors to execute our clinical trials – we don’t generally do the work in-house. I would love to report that one can rely on vendors and that they require minimal oversight but unfortunately this is just not the case. Strong governance is essential. I often get asked questions from colleagues or people outside my function, “Why do you need these people on your team? Why is that CRA or that clinical trial manager there? Isn’t the CRO running everything? Can’t they just do all the work for you?” I wish it were that simple! CROs are pulled in a lot of different directions. However much they care about your project, it will never be as much as you – it’s your baby! So, you need to be on top of the details. It’s not just a good idea for monetary reasons, the new R2 version of the ICHE6 GCP guidelines makes it abundantly clear that Sponsors are expected to not just hand over control of their trial to a CRO. Sponsors now must have a very clear and the systematic process to select and oversee CROs and be able to document “Why did we choose this vendor over that one? Or “How are we ensuring that the vendor is building quality into their work?” A lot of smaller Clinical Ops teams do not have these systems and processes in place or a good way to document CRO selection choices that survives succession and turnover.
Outsourcing sheds some work, but it also creates a lot of additional work for Sponsors and a lot of challenges to constantly ensure that the vendors are doing what needs to be done. It would be great if it worked like the airlines – here is the destination – get us there, but that just isn’t reality. The right systems can help. For example, in Clinical Maestro, the history of the vendor selection process and criteria is all organized, accessible and documented centrally. All the RFP and bid documentation is available in one place, so can prove how and why we selected our vendors. That’s just one of the ways that Clinical Maestro can help with these challenges.
For another example, we had a problem with a fairly large vendor that was set up on a sizeable time and materials budget. I normally prefer a milestone-based approach for larger projects, but my team convinced me that they had done this for previous trials and were comfortable with this arrangement. The problem subsequently came to light that the time and materials approach effectively added several hundred thousand dollars to the study cost. Even though we added nothing to the scope of work, the vendor came back to us every quarter and said “We spent a lot more time to do the work than we initially anticipated, so we need to charge another hundred thousand dollars. Here is a reconciliation invoice.” Because it was a time and materials agreement, we were obligated to pay. If we had a tool like Clinical Maestro’s LEAD module for accruals and forecasting using vendor-reported actuals, that could have been avoided because we would have had more insight before entering this arrangement and we would have seen the trajectory of variance to budget sooner instead of getting surprised.
Strategikon: When Clinical Operations wants to improve budget and forecasting, where should they look first? Why?
MJ: For budgeting and forecasting, the pain points or challenges again come down to information and how we can get the information needed to estimate how much a trial is going to cost. I’ve developed my own tool over time in Excel. It’s very simple and honestly a bit crude but it was useful as a starting point. Unfortunately, a lot of Sponsors start with the vendor bid as their forecasting tool. They may get even multiple vendor bids and take the average for their forecast. This is not good. One of the biggest reasons why this approach is suboptimal is that that does not allow you do to any kind of scenario planning or modeling. It’s hard to tell each vendor “I need 12 bids from you”. It’s hard enough to get a second scenario out of them. When you can run other scenarios, you can go back to senior management and say, “If we did this as an eight-week study it could be an $8 million study instead of $10 million study.” Clinical Maestro should solve many of these challenges; once you build the basic framework for the study, running scenarios is super easy – you can change one or two assumptions or work parameters and see how it takes you in the right direction, or not.
Strategikon: Does the size of one’s company provide benefits or put you at a disadvantage when dealing with outsourcing partners?
MJ: I’m not alone when I say that I definitely feel I’m at a disadvantage in the marketplace. As a small Sponsor I focus on working with CROs that are small to midsize and generally avoid the larger CROs. The big CROs naturally focus on landing big pharma clients who can generate tens of millions of dollars in revenue every year with multiple trials or programs. Small companies have a lot of trials in total, but they’re dispersed across different companies. In response, most large CROs have developed separate business units to try and better cater to the small companies. I think that’s good that they’re trying but I still believe that CROs allocate the top resources to the company that gives them the most revenue. Logical for the CRO maybe, but it’s a game I can’t win. For example: I was working with a larger CRO and needed to find more sites and implement a new strategy – finishing our trial early had become a huge priority for my company. The CRO had been after me for months to give them a chance to show what more they could do. I said, “Here’s your chance: if you can find me five new sites and get them up and recruiting in a short amount of time – then yes, I will be impressed and grateful.” The CRO team was excited and confident that they could deliver on this, but a couple weeks went by and … nothing. Over the next several months I exchanged emails and made calls and received more assurances, but fortunately I was working on other strategies as well. We were able to successfully finish the trial about three months ahead of schedule, but largely because we found a small patient recruitment vendor to make the changes that we needed.
When I work with smaller CROs I feel like they care more about us. As logical as it was for large CROs to focus on large pharma, our trial might be 5% of a small company’s business for the year, so we’re important to them. And I can set up a meeting with their CEO if there are problems and not get stuck in the bureaucracy.
These are some of the challenges that I’ve experienced working within an outsourcing model where we are highly reliant on vendors. From a small Sponsor perspective, we would benefit greatly from tools that give us better information: about trial costs, about vendor quality, about trial execution, and finally about the options if something goes wrong or changes. The right information is critically important to success!
About the Expert
Mark Joing is Founder & CEO of Mojo Trials, an emerging company offering virtual site monitoring for clinical trials. Prior to Mojo Trials, Mark was VP of Clinical Operations at Menlo Therapeutics and Nora Therapeutics, and he held management roles at Nuvelo, 3M Pharmaceuticals, and Abbott Labs. He holds a BS from Northwestern University and an MBA from the Stanford Graduate School of Business. Mark is passionate about making clinical trials better, faster, and cheaper.
