Chicago Sports
Confidential document reveals key human role in ShotSpotter gunfire detection system
CHICAGO — In more than 140 cities across the United States, ShotSpotter’s artificial intelligence algorithm and intricate network of microphones evaluate hundreds of thousands of sounds a year to determine if they are gunfire, generating data now being used in criminal cases nationwide.
But a confidential ShotSpotter document obtained by The Associated Press outlines something the company doesn’t always tout about its “precision policing system” — that human employees can quickly overrule and reverse the algorithm’s determinations, and are given broad discretion to decide if a sound is a gunshot, fireworks, thunder or something else.
Such reversals happen 10% of the time by a 2021 company account, which experts say could bring subjectivity into increasingly consequential decisions and conflict with one of the reasons AI is used in law-enforcement tools in the first place — to lessen the role of all-too-fallible humans.
“I’ve listened to a lot of gunshot recordings — and it is not easy to do,” said Robert Maher, a leading national authority on gunshot detection at Montana State University who reviewed the ShotSpotter document. “Sometimes it is obviously a gunshot. Sometimes it is just a ping, ping, ping … and you can convince yourself it is a gunshot.”
Marked “WARNING: CONFIDENTIAL,” the 19-page operations document spells out how employees in ShotSpotter’s review centers should listen to recordings and assess the algorithm’s finding of likely gunfire based upon a series of factors that may require judgment calls, including whether the sound has the cadence of gunfire, whether the audio pattern looks like “a sideways Christmas tree” and if there is “100% certainty of gunfire in reviewer’s mind.”
ShotSpotter said in a statement to the AP that the human role is a positive check on the algorithm and the “plain-language” document reflects the high standards of accuracy its reviewers must meet.
“Our data, based on the review of millions of incidents, proves that human review adds value, accuracy and consistency to a review process that our customers — and many gunshot victims — depend on,” said Tom Chittum, the company’s vice president of analytics and forensic services.
Chittum added that the company’s expert witnesses have testified in 250 court cases in 22 states, and that its “97% aggregate accuracy rate for real-time detections across all customers” has been verified by an analytics firm the company commissioned.
Another part of the document underscores ShotSpotter’s longstanding emphasis on speed and decisiveness, and its commitment to classify sounds in less than a minute and alert local police and 911 dispatchers so they can send officers to the scene.
Titled “Adopting a New York State of Mind,” it refers to New York Police Department’s request of ShotSpotter to avoid posting alerts of sounds as “probable gunfire” — only definitive classifications as gunfire or non-gunfire.
“End result: It trains the reviewer to be decisive and accurate in their classification and attempts to remove a doubtful publication,” the document reads.
Experts say such guidance under tight time pressure could encourage ShotSpotter reviewers to err in favor of categorizing a sound as a gunshot, even if some evidence for it falls short, potentially boosting the numbers of false positives.
“You’re not giving your humans much time,” said Geoffrey Morrison, a voice-recognition scientist based in Britain who specializes in forensics processes. “And when humans are under great pressure, the possibility of mistakes is higher.”
ShotSpotter says it published 291,726 gunfire alerts to clients in 2021. That same year, in comments to AP appended to a previous story, ShotSpotter said more than 90% of the time its human reviewers agreed with the machine classification, but the company invested in its team of reviewers “for the 10% of the time where they disagree with the machine.” ShotSpotter did not respond to questions on whether that ratio still holds true.
ShotSpotter’s operations document, which the company argued in court for more than a year was a trade secret, was recently released from a protective order in a Chicago court case in which police and prosecutors used ShotSpotter data as evidence in charging a Chicago grandfather with murder in 2020 for allegedly shooting a man inside his car. Michael Williams spent nearly a year in jail before a judge dismissed the case because of insufficient evidence.
Evidence in Williams’ pretrial hearings showed ShotSpotter’s algorithm initially classified a noise picked up by microphones as a firecracker, making that determination with 98% confidence. But a ShotSpotter reviewer who assessed the sound quickly relabeled it as a gunshot.
The Cook County Public Defender’s Office says the operations document was the only paperwork ShotSpotter sent in response to multiple subpoenas for any guidelines, manuals or other scientific protocols. The publicly traded company has long resisted calls to open its operations to independent scientific scrutiny.
Fremont, Calif.-based ShotSpotter acknowledged to AP it has other “comprehensive training and operational materials” but deems them “confidential and trade secret.”
ShotSpotter installed its first sensors in Redwood City, Calif., in 1996, and for years relied solely on local 911 dispatchers and police to review each potential gunshot until adding its own human reviewers in 2011.
Paul Greene, a ShotSpotter employee who testifies frequently about the system, explained in a 2013 evidentiary hearing that staff reviewers addressed issues with a system that “has been known from time to time to give false positives” because “it doesn’t have an ear to listen.”
“Classification is the hardest element of the process,” Greene said in the hearing. “Simply because we do not have … control over the environment in which the shots are fired.”
Greene added that the company likes to hire ex-military and former police officers familiar with firearms, as well as musicians because they “tend to have a more developed ear.” Their training includes listening to hundreds of audio samples of gunfire and even visits to rifle ranges to familiarize themselves with the characteristics of gun blasts.
As cities have weighed the system’s promise against its price tag — which can reach $95,000 per square mile per year — company employees have explained in detail how its acoustic sensors on utility poles and light posts pick up loud pops, booms or bangs and then filter the sounds through an algorithm that automatically classifies whether they’re gunfire or something else.
But until now, little has been known about the next step: how ShotSpotter’s human reviewers in Washington, D.C., and the San Francisco Bay Area decide what is a gunshot versus any other noise, 24 hours a day.
“Listening to the audio downloads are important,” according to the document written by David Valdez, a former police officer and now-retired supervisor of one of ShotSpotter’s review centers. “Sometimes the audio is compelling for gunfire that they may override all other characteristics.”
One part of the decision-making that has changed since the document was written in 2021 is whether reviewers can consider if the algorithm had a “high confidence” the sound was a gunshot. ShotSpotter said the company stopped showing the algorithm’s confidence rating to reviewers in June 2022 “to prioritize other elements that are more highly correlated to accurate human-trained assessment.”
ShotSpotter CEO Ralph Clark has said that the system’s machine classifications are improved by its “real-world feedback loops from humans.”
However, a recent study found humans tend to overestimate their abilities to identify sounds.
The 2022 study published in the peer-reviewed journal Forensic Science International looked at how well human listeners identified voices compared to voice-recognition tools. It found all the human listeners performed worse than the voice system alone, saying the findings should lead to the elimination of human listeners in court cases whenever possible.
“Would that be the case with ShotSpotter? Would the ShotSpotter system plus the reviewer outperform the system alone?” asked Morrison, who was one of seven researchers who conducted the study.
“I don’t know. But ShotSpotter should do validation to demonstrate that.”
___
Confidential document reveals key human role in ShotSpotter gunfire detection system Read More »
Making beautiful music together
Three factors determine the price of a violin, Mel Sapp explained, just as I was leaving the bright, airy shop she and her husband Greg run in Batavia: one is workmanship. Two, materials. And three, the name of the luthier who built it.
“You notice I didn’t say, ‘sound,'” she added. “Sound is subjective. You can change it.”
Indeed, most masterpiece instruments of old –by Amati, Guernari, Stradivari — have been modernized over the years, their necks and fingerboards lengthened, to bring them into line with current musical tastes.
I am not in the market for a violin, alas. But I visited Sapp Violins earlier this month because of a quip. When the shaky future of journalism is being discussed, with what colleagues I yet retain in a rapidly contracting profession, I’ll sometimes attempt to both sound a positive note and move the conversation along by observing, “They still make violins.”
Meaning, even antique trades thrive, for some.
Though it got me wondering: How is the violin business doing? Chicago, being home to one of the world’s great orchestras, is unsurprisingly also a center of violin craftsmanship. After I visited Sapp, the January Chicago magazine took an in-depth look at John Becker, the Fine Arts Building luthier to the multi-million dollar instruments of musical stars such as Joshua Bell, the article by Elly Fishman itself a finely constructed marvel.
So how does one get into the violin making biz?
Photo by Neil Steinberg
Greg Sapp was a music education major at Duquesne University in the mid-1970s when he had a realization that often comes to those whose ambitions lie in the arts.
“This isn’t going to work.”
Luckily, senior year, he had a class with the very 1970s name, “Creative Personality.” His final project was constructing an Eastern European folk instrument called a “prim.”
“It’s kind of like a mandolin,” Greg said, pointing to the ur-instrument, displayed on the wall. “I was the only one in my class that made something so functional.”
That wasn’t a complete accident — his father was a woodworker and singer.
Greg moved to Chicago in 1978 to attend the Kenneth Warren & Son School of Violin Making (now the Chicago School of Violin Making). He also bumped into Mel, whose car had broken down and needed a lift to the train station. When Greg told her he was going to violin school, Mel, who’d known her share of prevaricating creeps, assumed he was lying.
“How do I find these guys?” she asked herself.
Now Greg, 69, divides his time between building and repairing violins, and Mel does the books. Business is solid — they have three employees. Aubrey Alexander was busy at work when I visited.
Photo by Neil Steinberg
“I’ve always been more in tune with the violin, no pun intended,” said Alexander, explaining her choice of profession. “I don’t do well with people so much.”
How did she get started?
“When I was 8 years old my mom took me with her to pick up my sister’s violin when it was repaired. I was instantly fascinated by the tools and the instruments,” said Alexander, 39. “When I later started taking lessons I was always more interested in my cello and how it worked, rather than actually playing it.”
And what does it feel like, to create a violin with your hands?
“I start to associate a personality with the instruments,” she said. “They take on a personality of their own. I name the instrument. I gender them. This one’s a boy. That one’s a girl.”
How can she tell? A fraught question nowadays. It isn’t as if you can flip a violin over and check.
“It’s more about the feeling and how I interact with the instruments,” she said. “If it’s giving me a lot of trouble, it’s a boy.”
The vast majority of her instruments identify as girls. Her last cello, for instance, was named “Ophelia,” after the Lumineers song.
The Sapps also tend to anthropomorphize their instruments.
Violins “sulk.” They wait for buyers like puppies in a pet store. “Some instruments like kids better than others,” Mel said. The violins choose their eventual owners like wands in a Harry Potter book.
“The way I look at it, these instruments are all waiting for their person,” she said.
Working with stringed instruments is a protracted process — constructing a violin can take years (new projects tend to get put aside in favor of more pressing repairs, which themselves can take months). A violinmaker is seldom rushed. I wondered if hobbies are necessary and, if so, what Alexander does to relax from violinmaking. She told me she loves to fish, particularly bass fishing — she is from East Texas after all.
“When I’m not up to my elbows in wood shavings, I’m up to my elbows in lily pads,” she said. “Pretty much all I do is make violins, and I fish and make coffee.”
Speaking of wood. The top of a violin is spruce, the back, sides, neck and scroll are maple. The two types of wood, soft spruce and hard maple, combine to create an ideal sound. Along with a healthy dose of time.
While aging wood is important — Sapp pays hundreds of dollars apiece for small pieces of lumber that have sat for decades — everyone agrees that once constructed, violins need to be played to keep their sound fresh.
Playing “keeps it doing what it needs to be doing,” Greg said.
That sounds almost spiritual, I observed.
“Oh, This is juju personified,” Mel said with a laugh.
And on that note — sorry, couldn’t resist — we reach our fine, pronounced fee-nay, the musical term for the end of a composition.
Photo by Neil Steinberg
Making beautiful music together Read More »
Blackhawks stymied in 2-1 loss to Kings
The Blackhawks got a pretty clear reminder Sunday night about their place in the league.
After winning six of their last seven games, the Hawks lost 2-1 on Sunday to the Kings in front of 19,236 at the United Center. Jaret Anderson-Dolan scored twice for the Kings, while Petr Mrazek stopped 25shots to keep the Hawks somewhat in the game.
Mrazek didn’t get much help until it was too late.
Saturday night in St. Louis, the Hawks didn’t muster a shot on goal until Jason Dickinson scored 10:27 into the first period. On Sunday, it took the Hawks even longer to challenge the Kings’ Pheonix Copley, and Jonathan Toews’ shot off a centering feed from Patrick Kane with 4:24 remaining in the period was stopped by the Los Angeles goalie.
The Kings had no such issues.
Jaret Anderson-Dolan gave Los Angeles a 1-0 lead with 10:31 remaining in the first when he pounced on a rebound of a Kevin Fiala shot and hit a vacant net. Anderson-Dolan doubled the Kings’ advantage 6:07 into the second when he got free in the slot and beat Mrazek.
Finally in the third, the Hawks created sustained pressure, which paid off when Ian Mitchell scored his first of the year at the 16:45 mark of the third. That pressure also drew a Drew Doughty penalty at the 19:37 mark, but Los Angeles killed the abbreviated power play.
The Hawks also had to play the last two periods without forward Tyler Johnson, who didn’t return after the first intermission due to an undisclosed injury.
Blackhawks stymied in 2-1 loss to Kings Read More »
Healthy Patrick Kane impressing Blackhawks coach Luke Richardson with subtle plays
Sunday night against the Kings was Patrick Kane’s fifthgame back in the Blackhawks’ lineup since missing threecontests with a lower-body injury.
Hawks coach Luke Richardson said he hasn’t asked Kane about his health a lot, and hasn’t seen the star winger shy away from anything on the ice. Most importantly, Richardson thinks Kane is 100 percent healthy.
“Keeping him out that extra few days helped,” Richardson said. “He’s busting to have a big game.”
Known for his offense, Kane entered Sunday with threepoints since returning, but has recently impressed Richardson in other ways. Richardson mentioned Kane’s backchecking, and talked about a play Thursday night in Philadelphia when Kane dumped in the puck, allowing teammate Tyler Johnson to retrieve it.
Plays like that are subtle but stand out to Richardson.
“That’s something that’s not in his DNA, but he knows how to win,” Richardson said. “To get him playing that way and sacrificing, going after those extra points or offensive chances, is a smart thing for us as a team and we’re lucky that he understands that and he plays [for] the team and [what] we need to win.”
Lafferty’s impactOne reason for the Hawks’ recent uptick has been forward Sam Lafferty. Richardson singled him out as an example for how to play, and praised him for his feistiness Saturday in St. Louis.
“That becomes contagious on a team, not just skating but playing like a team, playing together after whistles, everybody’s in the scrums and maybe dictating a little bit of that physical play, too,” Richardson said. “I saw him take a few runs at their defensemen, and he’s a big guy that can skate. It makes you feel a little uneasy back there when a guy like that’s coming. He’s quiet, he doesn’t get into verbal battles with guys on the ice. He’s just all business and I like everything about his game right now.”
Forward Max Domi sounded similar Saturday.
“Laff brings something, he’s usually our staple of skating, and he’s one of the fastest guys on our team,” Domi said. “When he’s playing with energy, it usually infects our whole group.”
Staying connectedEven with today’s modern communication technology, it seems like we’re more disconnected than ever.
That isn’t the case with the Hawks.
“Nowadays in the game of hockey or anything in the world, young people have social media, they have their iPhones and their iPads and their headphones, and they don’t seem to be connected together,” said Richardson, who was asked what has surprised him during his first season as a head coach. “They may come do their jobs together or come do what they have to do.
“This team seems like it’s an old-school hockey team,” Richardson added. “They like being together, they enjoy each other, they plan extra things with each other and I haven’t seen that on teams in a long time. That’s really not surprising, just pleasing for a coach.”
Healthy Patrick Kane impressing Blackhawks coach Luke Richardson with subtle plays Read More »
1 dead, 3 injured after vehicle strikes Chicago fire truck on Stevenson Expressway
A motorist was killed and three passengers hurt when their vehicle struck a Chicago fire truck early Sunday on the Stevenson Expressway.
The fire crew was responding to a crash about 2:40 a.m. in the southbound lanes of Interstate 55 at Pulaski Road when it was struck by a vehicle, Illinois State Police said.
The driver of the vehicle, a 22-year-old woman, was taken to Mount Sinai Hospital and died from her injuries, Chicago fire officials and the Cook County medical examiner’s office said.
Three passengers, two men and a woman, were taken to the same hospital in serious-to-critical condition, fire officials said.
No further information was immediately available.
1 dead, 3 injured after vehicle strikes Chicago fire truck on Stevenson Expressway Read More »
High school basketball: Michael O’Brien’s Super 25 high rankings for Jan. 22, 2023
Occasionally the Super 25 surprises me. This was one of those Sundays. There were major upsets all over the area this week and it appeared that making sense of all of it was going to be a herculean task.
That wasn’t the case. Everything fell into place fairly easily. This was a good week to take a loss. With so many teams dropping games everything sort of evened out.
Simeon is back at No. 1. That was an easy decision. The Wolverines have an excellent resume and have beaten Benet.
Kenwood only dropped a couple of spots despite losing three games. All three were to ranked teams, so there is no shame in that. And the Broncos have wins against Young, Joliet West, Proviso East, Bloom, Simeon and Curie. That’s a stronger resume than Benet. Wins matter more than losses in the Super 25.
Hillcrest jumped to No. 6 and almost went a bit higher. The Hawks’ only losses this season are to East St. Louis and Belleville East. Hillcrest hasn’t had a single let down loss and has knocked off Curie (when they were full strength), Oswego East, Marian Catholic and Bloom.
Libertyville falls out after losing to Warren. Downers Grove North, which has been in and out of the Super 25 this season, is back after beating Bolingbrook.
Loyola, which joined the rankings last week, lost to Taft on Saturday but holds on because it also beat Brother Rice.
Super 25 for Jan. 22, 2023With record and last week’s ranking
1. Simeon (19-1) 2Robert Smith is one win away from 500
2. Benet (22-1) 4Handled Kenwood
3. Kenwood (16-5) 1Three consecutive defeats
4. Joliet West (17-5) 3Justus McNair is back
5. Young (17-5) 3Lost to Joliet West
6. Hillcrest (21-2) 8Unbeaten in the area
7. Brother Rice (20-3) 7Ahmad Henderson tough to beat
8. Rolling Meadows (20-3) 6Lost to Brother Rice
9. Hyde Park (18-4) 23Beat Kenwood
10. Mount Carmel (19-2) 10Hosts St. Rita Tuesday
11. Bolingbrook (16-6) 11Up and down week
12. Hinsdale Central (19-3) 19Faces Lincoln-Way East Saturday
13. Oswego East (18-5) 15Playing very well
14. Proviso East (16-4) 12Hosts Proviso West Saturday
15. Curie (13-9) 5Forfeits and suspensions
16. Lyons (18-3) 16Lost to Hinsdale Central
17. St. Rita (13-8) 18Big test vs. Mount Carmel
18. Glenbrook South (18-4) 20Handled Fremd
19. Lincoln-Way East (18-2) 13Lost to Bolingbrook
20. Bloom (14-6) 14Lost to H-F
21. Glenbrook North (20-2) 22Knocked of New Trier
22. New Trier (18-4) 17Hosts Glenbrook South Friday
23. Downers Grove North (18-3) NRCan play with anyone
24. Marist (18-5) 24Fell short vs. Brother Rice
25. Loyola (18-6) 25Beat Brother Rice, lost to Taft
High school basketball: Michael O’Brien’s Super 25 high rankings for Jan. 22, 2023 Read More »
The day was December 18, 2022 and the Chicago Bulls had just been embarrassed by a Minnesota Timberwolves squad down both Karl-Anthony Towns and Rudy Gobert.
The Wolves laid 150 points on Chicago in regulation, marking the first time the Bulls had given up that amount of points in 40 years. It was inexcusable. To say it was a humbling effort would be an understatement.
Following that loss, the Bulls had some lively conversations in the locker room. There was finger-pointing. There likely was a bit of animosity present.
But then, Goran Dragic spoke up.
“We’re not playing for each other,” Dragic said.
Those were the words that might have changed the outcome of this season for the Bulls.
Since Goran Dragic called out the Chicago Bulls’ realities, the team is 10-6
“So I think when Goran talked about we’re not playing for each other, you’ve got to pick yourself up off the mat because it’s a game that is imperfect. And you’re going to be making mistakes throughout 48 minutes. It’s about how you collectively respond to overcome what’s in front of you regardless of what is going on. The more we can play and care for each other, regardless of what you’re going through individually, that’s what can create the consistency we need,” head coach Billy Donovan said.
Donovan hit the nail on the head. The Bulls looked like a team that was far from living in the moment. Zach LaVine, especially, had gotten in his own head for the majority of the season leading up to that point.
You could tell he wasn’t mentally the same guy. But then, Dragic’s words rang through, and now the Bulls have had a winning record since that historical loss, and LaVine is playing much better basketball.
In LaVine’s last 10 games, he’s averaging just under 28 points per game with 5.6 rebounds, 4.3 assists and even 0.7 steals, showing more effort defensively.
As the current 10 seed in the East, the Bulls are far from out of it. There’s just a handful of games separating them and the four seeds above them. With half the season to go and it seeming more and more apparent that they’ll stay put at the trade deadline, the Bulls could make a late season push with this newfound life.
Should Chicago get into the playoffs, though, they’ll have an uphill battle against the Eastern elite. Granted, they have played well against the top teams in the East this year, so maybe we’ll see a different story from a season ago.
The Chicago Blackhawks are starting to run into a problem. They are winning way too much now. They should get back to losing so they better their chances of the generational superstar Connor Bedard.
The winning continued on Saturday night when the Hawks paid a visit to their biggest rival, the St. Louis Blues. They defeated them by a final score of 5-3.
The Hawks got off to an impressive start in this one as they went up 3-0. Jason Dickinson, Sam Lafferty, and Reese Johnson all scored goals to get the Chicago lead up to three somewhat quickly.
54 seconds after Chicago went up by three goals, however, Jordan Kyrou scored for the Blues to cut the lead back down to two. In order to save the momentum, Andreas Athanasiou scored about a minute after that Kyrou goal to get the lead back up by three.
The Chicago Blackhawks put together a good game to win on Saturday.
It would be easy to just kind of give up when things get hard or pressure filled but they handled it well. They are not good by any means but there are some good habits being formed right now. Not rolling over for the opponent is one of them.
Later in the same second period, Ivan Barbashev ended up scoring one to get the Blues back in the game again. By this point, people had to be wondering if the Blackhawks would find a way to let this lead slip away.
In the third period, former Blackhawks defenseman Nick Leddy scored for the Blues and at this point, they were within one. As mentioned before, this felt bad at the time. They needed a hero to put the game away for good (if you were cheering for them to win instead of tanking).
Well, Max Domi scored a goal with four minutes left in regulation to get the lead back up to two. This goal was not an empty netter. It was a normal goal assisted by Tyler Johnson and Patrick Kane.
It was a good team effort but the big story was the rookie in the net. Jaxson Stauber made his NHL debut with the Blackhawks and was really good for it being his first career start. He has spent this season with the Rockford Ice Hogs after being an undrafted player out of Providence College.
Stauber made 29 saves on 32 shots to earn his first career NHL victory in his first career start. It was a great moment for him and his family as he has been working for this his whole life.
Yeah, the Blackhawks need to start losing again for a different reason but it was nice to see his teammates score five in this one to help him to his first victory. Maybe he will find a way to have a nice NHL career. This was a good start.
Now, the Blackhawks have another game on Sunday night against the very good Los Angeles Kings at the United Center.
