Even when Siri is turned off when iPhone scree…

Relevant SMBC

I understand that, in many cases, eerily relevant ads can be explained by the Baader-Meinhof effect (which is related to frequency illusion), coincidence, and the fact that our internet activity can be used to infer many things about us that we did not reveal directly. But I recently had several alarming experiences where I could not find good reason to accept the above explanations. So I decided to perform several tests to see if I could more definitely isolate the cause for these ad-matches (if there was any).

Test setup:

In the course of these tests, there were no apps that had microphone access in my iOS settings menu. Facebook has never been installed on this phone. Siri was not completely shut off in my settings, but it could only be used when the screen was unlocked. (More on that later.)

It’s also highly unlikely that my phone was compromised by malware. I visit pretty run-of-the-mill websites, and I only installed a few well-rated apps and the rest I used were part of Apple’s standard suite.

I made sure there was no other hardware nearby that had microphones other than the iPhone. That includes IoT devices that smart TVs and the like.

I specifically chose keywords that met 3 conditions:

(1) I had not knowingly expressed any interest in (through typing on the internet, on the phone, in face-to-face conversation, etc.).

(2) were things that were specific enough that, if I saw a sufficiently closely related ad, I would know it was not a false positive.

(3) were the kinds of products and services that were internet-ad-friendly. For example, an internet GEICO ad is unsurprising, but historical manuscripts about botany, even if they are bought and sold by various collectors, aren’t usually the kind of thing that people and companies put up widespread internet ads for.

I would say the keywords near the phone and then wait about 10 minutes in case there was any schedule whereby new batches of audio recordings would be processed and turned into recommendations that would be sent over various third-party ad networks. I wouldn’t do anything online during that interim to ensure that I didn’t muddle or confound the results with other things that I was searching, clicking on, etc. After the 10 minutes had passed, I would refresh reddit’s front page or similar sites and keep track of how many times I refreshed before I got an ad that matched. Naturally, if I clicked to refresh 1000 times before getting a match, then the evidence that the ad was caused by surreptitious recording would be much weaker. However, given the fact that reddit’s font page has only two ads (at least for me when I disable browser extensions), there are not as many chances for a relevant ad to purely coincidentally show up in the first few refreshes.

Test 1:

I said to my screen-locked phone: “amazon kindle", “amazon”, “health insurance”, and “insurance”. I got some geico and generic amazon ads on reddit, but only after a lot of refreshing, so it could still be a coincidence.

Test 2:

I then said: “vacation”, “travel”, “expedia”, “travelocity”, “caribbean”, and “scuba diving”. I then refreshed reddit and I got 1 ad about insurance and the next ad was about studying in the Caribbean. That was the first red flag.

Test 2:

Immediately after that, I again spoke to my screen-locked iphone: “investments”, “mutual funds”, “retirement”, “stocks”, “bonds”, and “equity”. I waited 10 minutes again, and as soon as I refreshed reddit, I started getting quicken ads, ads about accounting, and ads about digital asset management. Not as strong of a match, but still worth mentioning.

Test 3:

The next day, I said: “clothing”, “jeans”, “pants”, “suits”, “shoes”, “boots”, “Nordstroms”, “JC Penny”, and “Marshalls”. I waited about 10 minutes, then refreshed reddit. The first time I got ads for a hooded sweatshirt that I had seen previously many times, so it might have been a coincidence. However, after the next refresh, I got an ad for Taylor & Stitch, a men’s formal fashion store that I didn’t even know existed before. Second red flag.

Test 4:

Next, I said: “motherboard”, “acer”, “asus”, “graphics card”, “electronics”, “television”, “computer monitor”, “computer screen”, “amd ryzen”, “amd cpu”, and “computer”. It is interesting to note that I repeated the televivision/screen terms at least once more, because the second time I refreshed reddit, I got an ad for some HD LG 4k computer monitor. This was a third red flag. I hadn’t seen any ad for anything close to a computer monitor (or any other consumer electronics like headphones, laptops, etc.) during all these tests over the last few days, so it’s very implausible that this showed up by random chance right after I mentioned it.

It’s interesting to point out how several of these matches would happen after the second refresh. It’s worth keeping in mind that ad networks are probably aware of the creepiness factor and want to minimize it so that the creepy feeling doesn’t prevent you from click on an ad, so it is plausible that they try to design the system so that the matched ad shows up recent enough so that it is relevant to your current interests but also delayed enough so that it doesn’t feel creepy but rather a natural, serendipitously relevant recommendation.

Analysis:

The only plausible explanation I can think of for these results is that the phone is listening to background audio when the screen is locked, even if (according to the iOS settings) Siri should be inactive at that time.

A crucial detail is that, after these tests, I decided to turn Siri off completely, and the ad matches seemed to disappear after further testing. That strongly suggests to me that, even if I set Siri to be "inactive” when the screen is locked so that it won’t respond to “hey Siri”, it is still not only listening but actively data-mining all background audio.

Maybe Apple is doing some kind of A/B testing or something. In any case, it is not implausible to think that they did a cost-benefit analysis and decided that the ambiguity of having Siri technically “on” but “inactive” on the locked screen was good enough legal barrier such that they could get the benefit of all that sweet audio data without too much liability.

I have to say that I was genuinely surprised with the results and disappointed with Apple. I understand that, when I use Siri, that audio may be processed for ads, but when I adjust the settings of my phone so that all voice-related features are off when the screen is locked, I felt that I had a reasonable expectation of privacy. I know Apple is no saint when it comes to privacy, but I didn’t think it would go this far. What next, will the camera take pictures and process them for targeted advertising, even when the screen is locked, just because the light-sensor software might still be running at that time?

I’m no expert at this stuff; I just thought I would share my experience. Was this already well-known among more savvy iPhone users? Have other people experienced this? Did I make some error in my analysis?

Found here