Every Mac comes with an (undocumented) family member classifier
Every Mac since macOS 13 (2022) ships a pair of daemons whose only purpose in life is to build a graph about who you are, who you know, how those people are related to you and to each other, how you communicate with them, where you go, what you do and when. In this post, we look into how this system works and give you a tool to peek into your own graph.
A family fork: one knight on e2 attacks White’s king, queen, rook and bishop at once. White’s pieces are the household the graph is about; the knight is the store, one move from every one of them. White’s own last move, 18.e3, emptied the square it stands on. White is in check and has one legal reply, Kh1; then the knight takes the queen. The check falls on the one who accepted the terms. What is lost is someone else.
3r1rk1/pp2qppp/8/3p1b2/8/1NQ1P1B1/PP2nPPP/2R2RK1 w - - 1 19
What if I told you that every Mac since 2022 ships a pair of daemons whose only purpose in life is to build a graph about you and the people around you: who you are, who you know, how those people are related to you and to each other, how you communicate with them, where you go, with whom, what you do and when. This graph is built on device by cross-referencing pretty much everything on the machine: Contacts, Calendar, Photos, Messages, Mail, location, Screen Time and a long tail of behavioral signals. And, more importantly, it is held as a queryable store that LLMs retrieve from (allegedly). Would you believe me?
Well, that’s what is happening, and interestingly enough, there is little documentation of how this whole mechanism works on your Mac. The closest thing to an official description is the man page (man 8 knowledgeconstructiond), and this is all you get:
Previous work: To be fair, Apple’s own researchers did publish a paper on the techniques behind this graph construction, back in 2023. But there is next to nothing on how it is integrated into macOS: which versions run it, and the exact list of what is collected and inferred. And not a word about the “kinship” inference discussed below. Also, we personally ran into it while doing some security research on the Apple Intelligence bundle in macOS 26 in August 2026; but a few months before, Nic Lydon came across the same database and wrote it up in I Built a Personal Knowledge Graph. Apple Had Already Built One on My Laptop. And even more recently, a forensics tool went looking on the phone: in September 2026, Alexis Brignoni taught iLEAPP to read the same store on iOS, where graph.db and the ontology.db that decodes its predicate codes sit under /private/var/mobile/Library/IntelligencePlatform; he writes it up in Reading the iPhone Knowledge Graph, and Checking Its Work.
But we go a bit deeper.
This blog post is an account of some analysis work we carried out on what the knowledge graph actually does, so that you (the user) can get a better understanding of how this mechanism works, and some clarity on what the graph holds about you and your family and friends. Everything in it was collected on macOS 26 (26.6.2 at the time of writing).
To this end, we open-source a local dashboard (iKnow) you can use to read the content of your personal knowledge graph on your Mac.
Finally, we derail a bit and look into what we do best: weaponizing and exploiting. We run a thought experiment (one that applies to any such personalized system) and show what happens if such a knowledge base ends up in the wrong hands, and how a “privacy single point of failure” can be exploited to threaten you in ways you may not have considered before (when combined with an abliterated LLM).
The two daemons keep their work in ~/Library/IntelligencePlatform, a directory of SQLite databases (fifty-one databases) stored in plaintext but located in a zone protected by TCC Full Disk Access.
Likely, the most interesting is graph.db. It is a triple store written in a vocabulary of 181 entity classes and 278 predicates: every row is a subject, a predicate and an object. The subject is an entity (a person, an app, an event); the object is either a plain value (a name, a handle, a date) or another entity, which is where the graph gets its edges. The predicate is the interesting part: it says what kind of relationship the system believes holds between the two.
One edge of the graph (synthetic data): a person and the organization the store holds as their employer. PS500 is the id predicate.json gives employer, and its range is not an organization but a relationship type — so one edge is two rows sharing a relationshipId, one naming the kind of relationship and one pointing at the other entity. The forty-six relationship type classes in the table below exist to carry exactly that, which is what lets an edge hold a confidence of its own and be retracted later.
Most of what the two daemons do is populate those predicates, and the tables kept beside them, from everything the OS will let them read. Into the graph go Contacts; Photos (faces); Calendar (titles and attendees); Family Sharing (including the age of every member); the Medical ID emergency contact in Health; events and contacts Siri Suggestions finds in Mail and Messages; people donated by apps through App Intents; and installed apps. Beside it, in tables of their own, sit who appears with whom in your photos; the communication log (who, when, which app, which direction; no content), which also pulls from your other devices; location and movement; and a long tail of device signals (screen locked, Wi-Fi, Bluetooth, charging, ambient light, and a stream named ContextualUnderstanding.PhysicalSocial whose record format lists people by name, phone and email, each with a start and an end time; it reads like a log of who was physically around you, though we never saw it filled in).
The complete list of the 181 entity classes:
Every class the shipped ontology declares, grouped rather than left in file order: the core types, then people, attributes, places and events, then the activity block, and last the forty-six relationship types, which are not things at all — each one reifies an edge so that the edge can carry facets of its own, a confidence among them. The ids and labels are Apple’s; the grouping is ours. intValue runs 1 to 185 with four numbers missing — 18, 54, 169, 181 — and nothing on the disk says what they were.
Fifty-seven of the 181 describe an activity: sleeping, exercising, mindfulness, going to bed, commute, dining, most of them with a matching type class beside them, so nearly a third of the vocabulary went on what you are doing rather than on who or where you are. That said, it is not clear that every class is actually used; some may be placeholders that are declared and never filled. In the table, we flag as implemented the classes for which we could locate a mechanism that might be responsible for the class, but those are to be taken as inference work rather than ground truth.
And this is the complete list of predicates (same disclaimer applies):
Every predicate the shipped ontology declares. The first column is the id as graph.db stores it; the last says how a value gets there: copied from a source, joined from another record, derived by a rule, shipped by Apple as world knowledge, or produced by a model (in bold). Seven we could not establish, and the table says so rather than guessing. The ids and labels are Apple’s; the plain-language readings are ours.
The methods run the whole range. Most of the vocabulary is filled by plain copying (a name from Contacts, an attendee from Calendar) and rules over the copies; the interesting rows come from models: a pair of XGBoost classifiers decides who is family from 144 mostly behavioral features, two embedding models are one of ten signals the system uses to decide when two records are the same person, twelve analyzers in Photos score partner, parent, child, coworker and friend from who appears in your pictures with you, on which trips and holidays, and Apple’s on-device LLM reads your messages to put a name and a relationship on a face (if Apple Intelligence is active).
Whatever the method, every row lands in the graph with a confidence attached, a number between 0 and 1.
The family scores live apart. Two CoreML models ship next to the ontology, EntityTagging_Family.mlmodelc and EntityTagging_FamilyAndFriends.mlmodelc: binary XGBoost classifiers that take 144 features and return a class and a probability over it. Their scores do not go into the triple store but into a table of their own (entityTagging.db, one row per person, seventeen relationship columns from family to mother to coworker, recomputed every day), which is what the iKnow scorecard further down reads. Only two of those seventeen columns have a shipped model behind them; what fills the other fifteen, and how any of these probabilities would become a confidence in the graph, we could not trace.
The feature names are in each model’s metadata.json, in plain text. They are the clearest statement of what this system is that I have found anywhere. All 144:
Every input EntityTagging_Family.mlmodelc reads, in the order its metadata.json declares them. EntityTagging_FamilyAndFriends reads the same 144 in the same order, and every one is a number. The groups are the schema’s own three blocks: twenty on who the person is, forty on when and how you communicate with them, eighty-four on the photographs you both appear in. Two of the twenty, photosPersonAge and photosAgeDifferenceCategory, are read from Photos too. The names are Apple’s; the group labels are ours.
The identity group is the tidy story: same surname, same home address, an iCloud Family flag — and it is the small half. The bulk of the model is the other two: who you message and who appears in your photographs, conditioned on the day of the week and the time of day, and then on both at once. photosPersonPosteriorProbabilityGivenSaturdayNight is a shipped feature name in a shipped classifier whose job is deciding whether someone is family.
Deleting something does not delete it everywhere.
The live graph forgets. Delete a contact and nothing re-derives them; the next run replaces the table wholesale.
But the store keeps a second table, expired_stable_graph, beside the live one. The retracted row moves there and keeps its confidence and its sources. How long it stays there is not clear from what we could see. It seems to go after about 48 hours (the store keeps a prune watermark, and a recent snapshot of the table reached back no further than that); but on my own Mac, I could still read very old records in it.
And a conclusion can outlive the evidence for it. On my Mac, the scorecard still scored two people the graph had dropped the day before, both of them known to it only from a calendar invitation. Well, if you want your partner not to discover some contacts you might have deleted and their “closeness” to you, better to drop some rows from expired_stable_graph too (unfortunately, you have to do it manually). But I am sure you have nothing to hide (just joking).
Is any of this sensitive? We’ll leave that to you, since “sensitive” means different things to different people (though I think you have plenty to choose from here). However, the thing is, none of it lives in isolation. It is built to be joined and correlated; the nodes are there to be linked:
Every node is a row in the store, and the record under it is what the store holds. Size is how many rows name it; shape is what kind of thing it is; the one node in accent is you, and a ring marks the four the store scores as family. A dashed line is a conclusion nobody typed.
Click a node — or reach the graph with Tab, walk it with the arrow keys and press Enter — and its record replaces these notes.
apps carrying this chat
apps carrying this chat
apps carrying this chat
apps carrying this chat
apps carrying this chat
(a chat kept by its handles)
apps carrying this chat
Click a node and the record under the graph is what the store holds on it (the rows, not a summary of them). The sixty-one faint ones are the part nobody expects: people the store knows and has not placed, in it because they wrote to you once, or because they were on somebody else’s invitation.
The most important point, though, is that this is not only about you (the person using the Mac). The people around you are in it too: your family, your friends, “anyone” you have talked to through your device. None of them was asked, and the store links them to you and, through the events and groups they share, to each other. A relationship is intelligence one level above the rows it was inferred from. For instance, by correlating across my own store, I (well, an LLM working for me) could easily work out the following:
It is easy to see how this could be weaponized if this single point of failure for your privacy ends up in the wrong hands. We play that (hypothetical) scenario out in a second write-up. The good thing is that this knowledge is meant to sta