It’s
been
just
over
a
month
since
the
former
Oakland
A’s
beat
the
Texas
Rangers
in
a
5-2
victory
that
closed
out
their
tenure
at
the
Oakland
Coliseum
—
and
indeed
the
city
of
Oakland
itself.
The
A’s
have
announced
plans
to
build
a
stadium
in
Las
Vegas
by
spring
2028.
If
all
goes
accordingly,
this
could
serve
as
an
opportunity
to
maximize
the
value
of
the
franchise
while
eliminating
the
challenge
of
attracting
star
talent
to
a
small
baseball
market.
The
A’s
featured
heavily
in
last
month’s
column
too
as
I
introduced
the
importance
of
benchmarking
and
selecting
the
appropriate
metrics
to
ensure
the
outcomes
and
goals
desired
are
achieved
using
the
movie
“Moneyball”
for
guidance.
As
depicted
in
that
movie,
former
Oakland
A’s
manager
Billy
Beane
embraced
a
different
strategy
to
get
around
the
small
market
problem:
Measure
the
ability
to
get
on
base
to
ensure
enough
runs
were
scored
to
win
enough
games
to
make
the
playoffs.
Of
course
what
was
once
a
competitive
advantage
is
now
known
by
all.
But
before
any
tool
—
be
it
analytics
or
GenAI
—
can
be
put
to
good
use,
you
need
to
understand
your
own
goals.
This
will
greatly
inform
the
tactics
and
resources
utilized
to
close
the
distance
between
objective
and
outcome.
What
Represents
Success?
For
law
departments,
success
may
be
supporting
an
aggressive
corporation’s
acquisition
strategy.
For
highly
regulated
businesses,
success
may
be
compliance
with
increasing
regulations
or
staying
out
of
controversy.
And
for
stable
businesses,
the
focus
may
simply
be
on
efficiency,
productivity,
or
cost
management.
Different
companies
pursue
different
strategies
and
those
strategies
should
inform
the
goals
of
the
law
department.
Goals
for
law
firms
may
vary
too.
One
firm
may
focus
purely
on
partner
profits.
Another
may
focus
on
growth
and
the
acquisition
of
new
clients.
A
small
firm
may
be
looking
to
be
acquired.
And
a
personal
injury
firm
may
look
to
increase
the
percentage
of
clients
that
receive
a
monetary
award
so
their
reputation
grows
and
their
marketing
is
more
effective.
For
a
legal
innovator,
it
is
important
to
understand
what
success
look
likes
for
the
larger
organization.
Apply
GenAI
To
Support
Success
Baseball
is
a
sport
with
limited
change
and
well
over
a
hundred
years
of
metrics
and
analytics
available.
But
we
are
in
the
very
early
days
of
the
GenAI
revolution.
There
aren’t
hundreds
of
tried-and-true
metrics
—
that
is
a
struggle
for
legal
innovators
right
now.
To
extend
the
baseball
analogy,
we
are
in
the
early
innings
of
one
of
the
first
games
of
the
season.
So
recommended
KPIs
and
guidance
on
metrics
are
likely
to
change
as
the
industry
learns
and
technology
matures.
Selecting
and
prioritizing
the
best
projects
and
solutions
to
support
the
organization’s
goals
is
important.
Just
because
there
is
a
new
bright
shiny
object
doesn’t
mean
that
it
supports
the
goals
of
the
organization.
Shohei
Ohtani
is
one
of
the
most
electric
baseball
players
in
modern
history.
He
can
pitch
and
he
can
hit
the
ball.
He
has
50
stolen
bases
and
50
home
runs
this
year
and
helped
the
Dodgers
clinch
the
2024
World
Series.
Any
baseball
team
would
love
to
have
him
on
their
roster,
but
does
his
production
map
to
success
at
any
team?
Probably,
as
Ohtani
is
a
once-in-a-generation
player.
But
what
about
the
next
tier
of
really
solid
players?
It’s
more
like
“does
the
baseball
glove
fit
the
fielder’s
hand.”
If
your
problem
is
in
the
outfield,
you
need
an
outfielder
and
not
a
catcher.
In
the
same
way,
selecting
the
proper
solutions
to
support
goals
is
important.
They
must
align
to
the
goals
for
success.
If
the
goal
of
a
corporation
is
to
accelerate
sales
growth,
then
the
law
department
might
want
to
look
at
solutions
that
shorten
sales
cycles
during
contracting
rather
than
the
latest
e-discovery
tool.
And
for
a
law
firm
focused
on
client
satisfaction,
solutions
that
make
the
firm
more
responsive
or
that
create
better
work
product
may
be
more
important
than
solutions
that
simply
reduce
nonbillable
costs
or
that
improve
productivity.
Understanding
what
solutions
support
success
for
your
organization
is
essential.
Selecting
KPIs
For
Solutions
Productivity
metrics
tend
to
be
more
measurable.
Does
a
GenAI
research
tool
save
time?
How
much?
The
greater
challenge
is
to
understand
whether
a
better
result
is
achieved.
What
if
a
research
tool
could
help
discover
a
new
legal
argument?
Or
a
new
technique
to
comply
with
regulations?
Understanding
qualitative
metrics
is
important.
Winning
a
court
case
or
finding
a
creative
way
to
settle
a
dispute
may
have
more
impact
and
value
than
the
productivity
benefits
of
a
solution.
For
any
new
solution,
it
is
also
essential
to
understand
the
metrics
of
the
baseline.
What
if
a
firm
would
apply
the
same
standard
for
hallucinations
used
in
the
Stanford
study
to
first-
or
second-year
associates
using
traditional
legal
research
solutions?
Recall
that
the
Stanford
study
defined
a
hallucination
as
an
answer
that
did
not
accurately
reflect
the
current
state
of
the
law
even
if
it
was
mostly
correct.
Perhaps
GenAI
legal
research
solutions
might
be
even
more
attractive
if
the
rate
of
hallucinations
were
understood.
GenAI
has
spawned
a
slew
of
new
document
drafting
solutions.
Productivity
metrics
like
reductions
in
drafting
effort,
shorter
cycles
to
complete
a
document,
or
fewer
turns
in
a
negotiation
may
be
the
absolute
best
metrics
for
document
drafting.
But
it
is
also
important
to
note
that
some
solutions
are
genuinely
hard
to
measure
and
document
drafting
is
a
great
example.
How
does
someone
decide
if
a
clause
in
a
contract
is
better
and
measure
that
objectively?
It’s
possible
to
benchmark
against
“market”
for
some
clauses,
like
those
that
limit
liability.
But
I’ve
yet
to
hear
anyone
come
up
with
a
framework
for
evaluating
whether
a
contract
is
better
and
produces
a
better
outcome
or
reduces
risk.
Assessing
outcomes
for
GenAI
solutions
can
be
challenging
given
that
there
isn’t
yet
a
strong
body
of
data
to
help
legal
professionals
benchmark
performance.
Start
with
what
success
looks
like
for
your
organization.
Understand
what
your
baseline
metrics
are
for
current
processes.
If
you
don’t
know,
then
measure
them.
Prioritize
and
map
new
solutions
against
organizational
success.
And
consider
the
importance
of
quantitative
and
qualitative
metrics
when
evaluating
solutions.
It’s
truly
an
exciting
time
to
be
working
in
legal
technology!
Ken
Crutchfield
is
Vice
President
and
General
Manager
of
Legal
Markets
at
Wolters
Kluwer
Legal
&
Regulatory
U.S.,
a
leading
provider
of
information,
business
intelligence,
regulatory
and
legal
workflow
solutions.
Ken
has
more
than
three
decades
of
experience
as
a
leader
in
information
and
software
solutions
across
industries.
He
can
be
reached
at [email protected].