
Interviews
Philip Rathle, Chief Technology Officer at Neo4j — a leader whose work has transformed how the world makes sense of connected data.

|
CTO @ Neo4j
Philip Rathle, Chief Technology Officer at Neo4 — a leader whose work has transformed how the world makes sense of connected data. Philip joined Neo4j back in 2012 as the company’s very first Head of Product, guiding its growth from under $1 million to $150 million in annual recurring revenue and helping establish the entire graph database category. Over the next decade, he built and led teams of more than 200 engineers and product managers, evolving Neo4j from a single on-premise database to a rich cloud portfolio now used by most Fortune 100 companies. Now as CTO, Philip focuses on the next generation of intelligent systems by integrating graph technology with AI, advancing global data standards, and shaping the future of how humans and machines understand relationships.
In this interview, Philip Rathle unpacks the philosophy that shaped his journey leading one of the world’s most innovative database companies. He shares how his “Philip-osophy” guides product decisions at Neo4j—helping teams balance quality, autonomy, and long-term vision. Philip traces the evolution from relational to graph databases, showing how understanding data relationships unlocks modern intelligence. He also explores the rise of GraphRAG, where graph databases meet generative AI for explainable reasoning. From transforming how enterprises use data to defining the next era of AI, Philip reveals what it means to build technology that mirrors the real world.
References
Philip Rathle – https://www.linkedin.com/in/prathle/
Jing Conan Wang (Host) – https://www.linkedin.com/in/jingconan/
FounderCoHo – https://www.linkedin.com/company/foundercoho
Timestamp
00:00 - Highlights
01:21 - Introduction and Interview Focus
02:49 - Growing up as a first-generation immigrant
03:35 - Early loss and learning responsibility
04:55 - Philosophy of quality and stepping up
07:45 - Building product at Neo4j
09:59 - The "Philipposophy" framework
11:01 - Early career in consulting and databases
12:42 - Discovering graph databases in 2011
13:33 - The NoSQL movement and what it missed
20:34 - Why graph databases are faster
27:13 - Graphs are eating the world
28:45 - How Google built a trillion-dollar graph business
34:04 - Knowledge graphs in enterprise AI
39:18 - Architecture for AI applications
43:22 - The GraphRag manifesto
49:46 - Where reasoning happens in AI systems
53:00 - Outro & Preview to Part 2
Full Transcript
I saw examples of the largest Oracle rack cluster in the entire company that is just grinding and not able to fulfill queries. You take that same data, model it in Neo4j, and on one tenth the hardware you can run a thousand times faster. Philip Rathle, CTO of Neo4j didn't just stumble into understanding systems. I lost my mom pretty early when I was around 10 years old, and so as part of that, I had to take on a lot of responsibilities that other kids my age, you know, didn't learn until maybe much later. That early loss taught him something that would define his entire career. Notice the big picture of what's happening. Notice where there's a need and then just jump in, whether invited or not as appropriate to jump in and just get it done. That philosophy led him to graph databases. The same technology that powers Google search engine and Facebook's network, and now he's building the tools for the future of AI. No one gets a 600% marketing uplift for anything, but this is because graphs are just such a different and higher value added way of doing things. But in the age of AI, Philip sees something most people are. The world shows up as networks, hierarchies, and paths. If you're building enterprise AI, that means you as the AI builder, have the responsibility for the outcome. If it's a good outcome, that's great. If it's a bad outcome, that's not so great, and so how do you put the guardrails into place?
Today, we're joined by Philip Rathle, Chief Technology Officer at Neo4j, a leader whose work has transformed how the world makes sense of connected data. Philip joined Neo4j back in 2012 as the company's very first Head of Product, guiding its growth from under $1 million to $150 million in annual recurring revenue and helping establish the entire graph database category. Over the next decade, he served as CPO and built and scaled the product organization from 1 PM and 12 engineers to 25 PMs and 200+ engineers, evolving Neo4j from a single on-premise database to a rich cloud portfolio now used by most Fortune 100 companies. Now as CTO, Philip focuses on the next generation of intelligent systems by integrating graph technology with AI, advancing global data standards, and shaping the future of how humans and machines understand relationships.
In this interview, Philip Rathle unpacks the philosophy that shaped his journey leading one of the world's most innovative database companies. He shares how his "Philip-osophy" guides product decisions at Neo4j, helping teams balance quality, autonomy, and long-term vision. Philip traces the evolution from relational to graph databases, showing how understanding data relationships unlocks modern intelligence. He also explores the rise of GraphRAG, where graph databases meet generative AI for explainable reasoning. From transforming how enterprises use data to defining the next era of AI, Philip reveals what it means to build technology that mirrors the real world. Let's begin.
My parents moved to the US a few years before I was born, got married here. So part of my story and part of my background was growing up in a household, in a first generation immigrant household. And I think what that gave me is a sense that different people have different ways of saying things. Another part of my early childhood experience was my mom was a Montessori teacher and so I had the rare opportunity to up until eighth grade and I would say even through high school just learn for the sake of learning. I didn't have grades through eighth grade, so truly the only metric was just myself. Another one is independence, but also interdependence. I lost my mom pretty early when I was around 10 years old. And so as part of that, I had to take on a lot of responsibilities that other kids my age didn't learn until maybe much later, maybe even like in parenthood. I had two younger siblings while at the same time seeing how a lot of people from the community and family just came together to play a substitute role in some way. I was very close to my mom, but I also really benefited from and appreciated all the people around me that came into care. And one way that that influenced just the way I lived and ultimately later on the way I worked is to just notice the big picture of what's happening, notice where there's a need, and then just jump in, whether invited or not, as appropriate to jump in and just get it done so that everyone can move on.
You have been growing and learning and develop very unique philosophy for your personal side and as a child who has been through a lot of your early days is also very different environment. I wonder how those philosophy personal philosophies influence you in the later part of your career.
Yeah, so I guess one important thing that came about maybe during my teenage years is the sense of I read the Zen of Motorcycle, Zen and the Art of Motorcycle Maintenance back then. And like the big takeaway from that book that stuck with me is a sense of quality. And the punch line spoiler for that book is it's really about quality, quality of experience, quality of relationships, quality of intention, quality of results, but you know of course that one's a little bit finer. You don't need to sweat things that don't matter so much. You need to choose where to put your effort, but having at least a sensitivity to quality and what matters is something I carried forward into my work life up until now.
Another one, which is something I tell PMs that I've hired over the course of the years is particularly early product managers who might be careful about, okay, here's my swim lane. I don't want to tread on anyone else. That's a real thing. You don't want to tread on other people and their responsibilities. On the other hand, life working in a startup, how to argue life more broadly, provides lots of opportunities where if you keep your eyes open for it, you'll see that there are some gaps in areas that are maybe adjacent to what you're doing that need to be filled and where you can come in in a really positive way and add value. And that's a natural way in which I grew my responsibilities early in my career and which I think anyone coming into the workforce, anyone in any stage in the workforce can get more fulfillment out of their work. Because by doing that, that implicitly means that you understand the bigger picture. And if you understand the bigger picture, then you're like the bricklayer that understands that they're building a cathedral, not just laying bricks.
And I'd say another takeaway is just, you want to understand the big picture, but you just want to get things done and get obstacles out of the way. So one way I like to, you know, joke and describe product management, especially early stage, especially to start up is product management is about understanding the big picture, understanding the obstacles that lie in your way, and just getting those obstacles out of the way.
So you had a great philosophy. It's like comes from your personal journeys about the quality of the taste and stepping up to getting more responsibilities and also jump in to help. And also as a CTO of Neo4j, what is your philosophy of building product for the Neo4j?
Yeah, so Neo4j was the first product hire back in 2012. So I met our CEO while he was still in the process of raising his A round and was looking for someone to basically, that he could hand off the responsibility of building and managing product so that he could graduate into, like, he's PMing the company. This is like my product way of describing a CEO's job. And for a long time, for a while I was an individual practitioner and then grew the team ultimately to about 25 people, then changed roles to CTO, handed the reins to a successor CPO.
In product, there was a point in the journey where there were enough people and my style is I like to delegate as much responsibility as possible to others. It took a while for me to realize that that was a thing. Early on, I like doing everything myself. But then after a while, you realize that, okay, my product can be the team and my product can be the portfolio and how things work together. At one point, we were preparing for an offsite, trying to get the team aligned. My chief of staff at the time, who was both brilliant and had a good sense of humor said, well, maybe what we need now is a Philip-osophy where you can provide a full, what is your philosophy for the product, for the journey, for how we make decisions around short versus long-term, how we prioritize different parts of the portfolio, what the bar is for the definition of done for different parts of the product, ranging from database kernel to the clustering layer to the database as a whole to the service across different tiers and for open source versus commercial versions all the way up to tooling and APIs. So we went in with a Philip-osophy which...
Can you elaborate more?
Philip's philosophy for the product. So it's the things that provide a framework or some guidelines inside of which the team can work. And then by understanding those, everybody can get a lot of latitude because they know that if they follow these basic principles, they'll be in sync.
Hmm, got it. Who come up with that name?
That was my Chief of Staff.
It's a very brilliant word. Captures the essence.
I think it's good if you take it with a sense of humor.
Yeah. You shared like a wonderful, both your personal philosophy and also the philosophy as you are building the team in Neo4j, which you put in very interesting way, like your product is the team, which is such an awesome and interesting point. I wanna like dive deeper later. But before that, I wanna go back to the early part of your life and careers. Like you had amazing careers. Can you share a little bit about your early career and how that's impacting.
Sure. So where do I start? I spent broadly speaking, so a decade in consulting and then discovered and had a startup and that's that I co-founded. And then coming off of that, I discovered product management and didn't look back. So I've been between CTO and product management and solution architect most of my career.
So you enter your career, I heard you're working in a company called Anderson Consulting.
Yeah, which today you would know as Accenture. Back at the time they were called Anderson Consulting. They, at the time, I basically fell into IT. I'd been working with computers ever since I was very young and playing with like Apple II and Texas Instruments machines and did some programming early on and liked it, and landed at Accenture or then Anderson Consulting, did software boot camp learning C and like some 4GL at the time and landed straight into a bank doing like COBOL programming but also getting database programming with mainframe DB2 at the time and some IMS if you want to go way back to early hierarchical databases.
If you think about the beginning part of this whole journey, at which point of your career you started to realize that graph is such an interesting thing you need pay attention to?
Yeah, it was really on my first encounter with the very, very, very early beginnings of Neo4j back in 2011 when the company was still in the seed stage and the product didn't quite have a query language and it was more of a Java library and couldn't, so very early days. But at that time, those who were around in the early 2010s, this was an era where there were all of a sudden new database models coming out for the first time in, you know, since the 1980s when the relational database was invented. So it was 2011 when I first discovered Graph. And if you look back at that time, what was happening is the industry was, this was the birth of NoSQL. And NoSQL primarily, the earliest adoption was in, don't know, kind of came out of technologies like DynamoDB at Amazon and Google Bigtable, where you simplified the model so that you wouldn't have tables and joins and basically remove the concept and the primitive of a relationship from the model so that you could do simple queries at scale, which was necessary in order to do e-commerce at scale, which was becoming really big in the early 2010s. My reflection on it though was, I feel like something's missing because in my experience and at the time I'd spent seven years working at another company called Embarcadero Technologies that had about a dozen tools for working with relational databases. So those tools could all work with any database to do database management, modeling, development, performance tuning through a single pane of glass. Any database really meant any relational database management system. So when the key value column family document databases like the likes of Mongo came around...
So those are all the concepts from the NoSQL databases.
Those are all like forms of NoSQL. And what those all share is there's no relationship, no way to relate things to other things. It basically simplifies down so that I can store and retrieve. And then you also had the Hadoop data lake, ultimately like data lake house. Now, Snowflake, Databricks, BigQuery, and so on. But those technologies as well, even the ones that embrace relational like Snowflake, aren't so much about relationships in data, but they're more about storing data and doing the kinds of analysis and the kinds of statistical analysis and querying that largely looks at one kind of data and scans through. It does essentially search and pivot table type queries at scale. So what was missing in all of that is what about the relationships and data? And the question I always find myself asking is, well, if the world shows up primarily as networks, as hierarchies, and as journeys and paths, this is true of biology, ecology, payments, computers, the way people interact, the way disease spreads. Real world phenomena are effectively showing up as graphs or as simplified networks like hierarchies like HR. And actually when you model a hierarchy, an HR hierarchy, you find that it actually ends up being more than just a strict hierarchy because each person has their career history, which is like a journey. And you might have a direct report, but then dotted line into a mentor and dotted line into a community of practice and dotted line into a project and so on. And then have all these skills and other things. The supply chains is another form of hierarchy, bill of materials or complex products. A lot of car companies and aviation companies store the entire hierarchy of their products inside of Neo4j to do various kinds of analysis, supply chain and so on.
The real world definitely works more like a network, like everything is interconnected. And one question I have related to about databases is about the reasons why different type of database become a mainstream. So as I heard you talk about, initially everyone was doing relational databases. And then later there was a time, I think before 2010, everyone was doing NoSQL databases.
Starting around 2009. Yeah. So great question. And let me give you Philip's perspective on this. The relational database. So when you go to database 101 class, the first exercise you have is you'll take an order form and then you'll like, how do I turn the information that someone fills out in this order form either online or used to be on physical paper or typewriter, put it into a computer like how do I store in a database? So you have these normalization rules which will tear that data apart and put it into respective places to avoid redundancy. Then you have technology to rehydrate it and store and retrieve. The reason that that's the database 101 example is that relational databases were born at a time, so the mid to late 80s, where the big business problem at the time that computers were able to solve was business process automation. So take processes that were human beings like writing on paper forms and digitizing that. And so the initial use case for relational databases then became how to take a paper form and store it in a computer in a database management system specifically. I don't mean to say that relational databases can only do that, that they've evolved to do a lot. But that was the initial design principle and a lot of the architecture for solving that as a problem, all the way including the fact that you a fixed schema model which corresponds to the fact that paper forms are human generated, they're easy to understand, and they're fairly fixed and it solved that problem well, which happens to be very unlike the real world, which changes all the time, like, now you've got all infinite kinds of different kinds of data sets that I can bring in and bring together. You have a requirement now for data to be much more agile or for your database to be able to store different kinds of data. So, okay, I'm gonna try this today. Some of it worked, some of it didn't. You want a model that's adaptable. Then you relational databases, because they were designed to do relationships, but are fairly simplified level to normal deduplication normalization rules is that a join table and primary foreign key relationships ends up not being the best computationally efficient way or the most understandable way or the most flexible way of working with connected data. And this is where graphs come in is with a graph you basically can say I have my table with my things. You can spill those things out as marbles, store them in the database and then have other kinds of things that each exist independently and just drop in a relationship as an object is an instance between any two nodes that have a relationship, whether those nodes are similar kinds of things or different kinds of things. That gives you a combination of schema flexibility, adaptability. You also get this thing called index-free adjacency is the graph jargony term for this. But what that really means is, if you're building a database natively from the ground up to support the property graph model, you can actually store your relationships on ultimately on physical storage and in memory in a way that either end of the relationship has a pointer that points to the node at the other end. What this means is I can get from one node to the next, the next to the next just by chasing pointers along the relationships. You can do about two million of those pointer chasing operations per compute core per second on CPU.
Wow that's very fast.
Early on with Neo4j, we had few enough customers and was few enough community users that I literally could go out and talk to everyone. I saw examples of, here's the largest Oracle Rack cluster in the entire company that is just grinding and not able to fulfill queries because it's a multi-level hop that just takes way too long across a large data set and where you take that same data, model it in Neo4j, and on one-tenth the hardware, you can run a thousand times faster. And that's just purely the difference in physics between doing index-free adjacent pointer chasing and grinding away on index and through primary and foreign keys and join tables.
You point a very interesting way to understand the database, which is that the database is designed for a problem, for a business problem. So the mainstream database practices actually depend on what are common problems. And relational database was designed for business process automation. That was very popular in early 1980s, 1990s. In fact, one of the very early application was Excel. Everyone was using it.
It's like a mini database. On that topic, what do you think about the keyline applications that are driving graph database right now?
So graph is always something that I've seen as a very horizontal opportunity, which is part of the reason that I'm still here alive and kicking after 13 years. Early on there was more recognition that social networks were graphs. That was kind of like the early use case everyone assumed was a graph use case. And it's because of Facebook that Facebook publicized this. Now, of course, Google did graphs from day one, like what differentiated them when they released in 2000 was not just that they had the largest index of the World Wide Web, but that they used a graph algorithm, namely PageRank, to rank the results. So these are maybe two distinct early use cases. And you could say LinkedIn is like a professional network. But some of the early use cases I saw in Neo4j ranged from property pricing. So Marriott was an early customer and used Neo4j to recognizing that the way properties are priced is very complex and involves this complex hierarchy and to process it across efficiently across one of the most queried booking engines on the internet. You needed a more better response time and a richer model to support that. Another was routing, so another early customer was Deutsche Post, who at the time were just getting deluged with early days of e-commerce and the postal network wasn't designed to handle that, so they basically redesigned the routing algorithm using Neo4j. Turns out today most pharma companies use a graph approach and use Neo4j for doing drug discovery just by playing a game of connect the dots between ailments, all the different parts of the body and how they interact with each other through biology and all the different candidate drugs. Supply chain is a graph. Some big pharma companies, many kinds of companies use Neo4j for that. There's all kinds of AI stuff, maybe we can, I'll put a pin in and we can come back to. So if I generalize my experience, in looking at recommendations, fraud detection, anti-money laundering, anti-policing kinds of use cases. I keep coming back to the generalization that the world shows up as networks, hierarchies, and paths. And anytime I have a problem that involves one of those, if I haven't applied either graph analytics in some form or real-time transactional operational database approach to those networks which are graphs, then there's probably, in fact, there's almost definitely some kind of value that's completely hidden from me, where if you get a smart technologist who understands graphs and an enterprising business person who can think a little bit creatively and out of the box, they will often find an opportunity to make a discontinuous leap in productivity and value. I almost see this as like in the same way that Mark Andreessen described as software is eating the world. I've long had this thesis and through my own eyes seen like maybe a version in the database space of graphs are eating the world where ultimately anytime you have a non-graph way of doing something, for use cases that are valuable, there's almost always a way to apply graphs in such a way that you just get 5x value, 10x value. I've seen cases where a company will do marketing and get like a 600% uplift in customer engagement by using a graph approach. Well, no one gets a 600% marketing uplift for anything, but this is because graphs are just such a different and higher value added way of doing things.
Yeah, exactly. Like you mentioned, graph is eating the world. And one thing I would like to dive deeper into is that graph has been used in many applications, many companies already, although we don't have that very often. You give two examples earlier. One is Facebook. Like everyone understand Facebook is about social network. So it makes sense for them to use a graph. And you also mentioned about, in fact, Google use a lot of graph technologies, all of them from beginning to pagerank or later, and it's really a trillion dollar company built on top of graphs. Any insights about how graphs are used in that domain, like search engines?
Search engines actually are interesting in two ways. One is Google is truly a company that's built primarily on its search business though, GCP is doing extremely well and they have other money making businesses, but it became a trillion dollar company based on its search business and that is entirely graph based business. So it's page rank number one, which uses the topology of the graph and does graph algorithmic computation in order to rank individual pages based on the number of inbound links and then doing that iteratively so that the inbound links are weighted based on the number of inbound links to the inbound referring page. Then 12 years after that, like 2012, Google released the Knowledge Graph. So Google's Knowledge Graph refers to the graph of, what did these things actually mean? Not just using the topology of the pages, but semantic meaning. And so that when I do a search on Rio, you then have a choice between, well, did you mean Rio the casino, Rio the city, or Rio the movie? And you know, Google doesn't publish numbers, but there are people out there that try to reverse engineer how large it is, and last I heard it was maybe probably around 50 billion entries, and close to a trillion relationships. Then more recently last year, some of the more complex searches that are happening, like I remember a travel example being shown at Google I.O. where you can see you bring back a graph and you query the knowledge graph and that's in the background used with the LLM to provide a better answer, which at the time this term wasn't that used, but we now know that as graph rag, which is retrieval augmented generation with a knowledge graph, not Google's knowledge graph necessarily, not limited to, but anyone being able to put whatever part of the world, real or digital that they care about that's relevant to their AI application into a graph, using that as knowledge for your AI application, and then querying that either from your AI agent or have your model know when to make a call out to the graph via MCP. That's actually become a pretty big use and a pretty big trend, again, across pretty much anything. Coming back to your question, what are the use cases? We've evolved from there are some common naive well-known use cases like social and web search to where now it's really anytime you want to mirror your real world or mirror your enterprise and have let's call it a digital twin of that to serve as AI knowledge not just as memory but and the distinction I make between memory and knowledge is memory is maybe persisting some facts and sometimes you talk about like using vector embeddings as memory but vector embeddings can't be they're not human understandable first of all like you look at a vector and it's just a bunch of numbers what does it mean, it's only if you have three vectors that you can actually get any value and then it's comparing the distance between this one and that one. And then I get, that's one algorithm and one way to understand the words, but not the context around how those things interact with the world and not in a way that can give you explainability, not in a way that can give you a whole set of richer answers that come from graph pattern matching and computation, as well as having optionality around let's say statistical based algorithms, be they graph or otherwise like PageRank, is an unsupervised graph machine learning to deterministic queries like graph pattern matching. So that's a mouthful. Let me get back to you.
You actually mentioned such a great paradigm shift even like starting with Google in this example is like the first generation of their algorithm is based on page rank, which the page on the one hand is the first landing page, but on the other hand is also the web page. So the whole Google web search engine is based on the fact that there is this graph of web pages. And then you mentioned about like in around 2012, they started to have this concept called knowledge graph, or entity graph. They have upgraded our algorithms on top of that things. And for many of our audience, and they may not have the context of the graph databases on that, how do you explain to them to the concept of knowledge graph? What is a knowledge graph? And why having a knowledge graph would be helpful to improve the results of search engines or other applications?
It depends who I'm talking to. Let me explain it from two different perspectives. If I'm talking to a CIO CTO, I describe it as in the enterprise, there's just some new AI executive orders in the US that essentially say, look, we're not going to push liability or responsibility for AI results to the model providers, which is, I actually think that's a good decision in many ways. But the consequence for a CIO, CTO, and everyone down the chain, including the developer, is that means if you're building enterprise AI, that means you as the AI builder have the responsibility for the outcome. If it's a good outcome, that's great. If it's a bad outcome, that's not so great. And so how do you put the guardrails into place? So a knowledge graph is a way to bridge human and machine understanding inside of my AI application so that AI already is not just one LLM. In fact, this is kind of my pet peeve around how AI is regulated and a lot of regulators talk about AI is oftentimes people will talk about AI and I think even AGI depends on an assumption that you have one monolithic thing which is one giant LLM foundation model that will answer all questions. In the enterprise, it's very much not the case that AI applications ever, like almost ever, just use one model. It's big models, small models, multiple models calling each other, models calling out to tools, and so on and so forth. But so a component that you need is a database that's part of your stack that not only includes your data, but connects your data and can be queried by the machine, but then also fed by and understood by humans. And so a knowledge graph is a graph database and the graph model happens to be an ideal way, like it's structuring data literally the way the human brain structures data as nodes and relationships or as synapses and neurons and synapses. It's structuring data in a way that's understandable to humans and that is most efficiently processed. The lens itself too efficient processing of complex connected data. And that is schema flexible in the sense that you can just add data. So one example, let me give you a concrete example, is Klarna was in the news a lot the past year because they built this AI application and said, look, we unplugged 1200 SaaS apps, including Salesforce and Workday and many others because we built this AI application. Well, little known fact is that AI application is basically cursor with Neo4j underneath it. Cursor, of course, is like the development tool. They build applications using cursor, but it's basically saying, I'm going to take all the data from across the enterprise, connect it, connect it using a graph. It doesn't necessarily mean you need to take all the data in the enterprise and put it into your graph, but it's taking the important information about the key entities and the things that are central to your business, connecting them together, and then iteratively growing on it in a way that actually gives you data network effects and use case network effects. Where use case network effect is, once I have my data to solve a particular problem in the graph, oftentimes, that is enough of a critical mass of data that you can solve other problems that you didn't even intend to solve. So I've seen companies that started maybe with one department here, one department here, one department here that didn't know about each other. Then they found out about each other. And then you started building new applications every month that leverage all three data sets and create a cohesive greater whole.
Yeah, I hear like two key benefits of using the graph is like when we think about using the data, either we just use the data by putting them on a solid good thing which is like the model itself, but in the enterprise cases like you mentioned oftentimes, you know the small modules of the entities and there's relationship among them. And if you organize the information this way, it's more human understandable. People can understand that. And if you also organize this way, they can encourage collaboration and build small pieces together, putting them, combine them to create like bigger pieces on top of that. Because everyone now is talking about large language models and how they want to use this in enterprise cases, what do you think about architecture for good, like AI applications would look like from your perspective?
Definitely, agentic architectures that leverage big and small models together for different tasks, but that also use where the application has the ability to be able to get probabilistic results for problems where that's acceptable and appropriate. But deterministic results for, let's face it, some questions have an exact answer. We kind of forget about that and because we become so impressed and so enamored with what models can do. I'm continually impressed and more and more reliant on my day-to-day life by leaning on LLMs. But for the myriad of enterprises use cases that are out there, you actually want to use a combination of multiple new technologies and then as well as existing technologies and use a combination of the best of multiple worlds. Another way, and this actually also is a follow-on answer to the last question of how would I describe it to someone who's deep into AI and understands AI history, which like you do and probably a good part of your audience does, is neuro-symbolic AI was a big thing back in the day. I see knowledge graphs and graphs as a way of another newer manifestation of bringing back the marriage of deterministic and non-deterministic methods. Broadly speaking, I'd say AI needs some orchestration layer for your machines, but your AI orchestration, you need multiple model providers, need an eval framework, but you need one or more databases that can serve as memory. This is probably obvious to most of the audience, but the AI regulation oftentimes speaks from a standpoint of assuming that all the models have all the data. This might be true of all the world's knowledge in terms of a foundation model getting trained on everything on the public Internet as of the last time it was trained. Inside the enterprise, you're not going to get a foundation model that's not particularly...
It's a private data, yeah.
It's private data. That private data can often change from moment to moment and you want to deal with a customer based on or a partner or an employee based on the latest data. You also want to be able to apply security and privacy access controls. Those are all things that a database can do really well. Then you also want to be able to provide some explainability. As much as there are some efforts to understand how models quote reason under the hood, like anthropics, efforts around mechanistic interpretability. Not only those are very early, but those are also passive. It's like I'm gonna introspect a model that's gonna do its own thing as opposed to being able to understand, impose, and own the data. Inside of the graph, have the domain graph, which is the graph of things. You can also have graphs that denote rules in cases where there's a known rule set that you just want to abide by. There's a graph of just all the interactions and the events. You can say that's the memory.
And I saw that you also wrote graph rag manifesto. Tell us about what is a graph rag and what is the manifesto as well.
GraphRag is the simple notion that if I make a call-out to my knowledge graph, as we've been talking about, either directly from some agentic application through a framework or have a model do a call-out, that constitutes GraphRag. So it's retrieval augmented generation using a graph. The company that really put this term on the map was Microsoft with a Microsoft Research paper. There's actually a Neo4j implementation of that out on their GitHub repo and there's been progress since then. At the time they wrote that, however, we'd already had dozens of customers and hundreds, if not thousands of community members independently start doing this of realizing, hey, look, turns out I can get better results that are further, more and more over explainable. And maybe this was less obvious in the early days because it's more of a productionization problem but apply security and access controls using a graph and that often they came to the graph after they had tried everything they could with just models, small, large, fine-tune, post-trained, and then tried vectors and realized that they couldn't get the bar high enough to go into production. So Graph-rag is that trend and the reason for writing the manifesto was to me, I was just seeing the show up over and over and over again. And a lot of people were getting it and hearing and discovering it independently. So I felt there was a need to just call it out, you know, so that people weren't stumbling into it after having exhausted every other option, but actually understanding and realize upfront that hey this is a pattern. And if I understand that my application probably has, I think the highest correlation is if the stakes of an application are fairly high, stakes meaning high dollar value, brand and reputational impact of bad answers, if there's any kind of regulation around anything from data privacy to bias and so on, that if your stakes are higher, then you're going to have a higher bar. If you have a higher bar, probably you're going to need explainability or privacy and security, and you need better answers. It turns out, GraphRag is not just one technique, like is one pattern broadly, but you have multiple ways in which you can use the graph. You can start with a vector search. So use a vector search to go and find the node in the graph and then pull everything from around it, maybe several levels out and then send that back to the model. So then you're providing more of the context around that particular thing. That will give you a better answer. Still probabilistic, but better answer. You also have cases where you can say, well, I'm going to rank my vectors based on something like PageRank or community detection or some other graph algorithm. You also have certain cases where if a question involves a multi-level query and there's math, say supply chain risk or banks doing stress testing and doing some kind of asset calculations with chains of asset ownership. Or one of the world's largest employers is building an employee career journey graph as a graph rag application with Neo4j, storing employees all the viable career paths so that you could, in those cases, you can actually get the answer from the graph. Because if you use a graph database, not only can it store your knowledge graph, which you can look at as being a fairly passive activity. But there are different kinds of, there's a query engine and analytic engine that you can use to actually ask questions that you could say maybe is a graph AI that complements your generative AI. So you can also delegate certain kinds of questions to the graph and have them answered by the graph database and then return that back and pass it through the model. Then there are other things like GNNs where, pretty popular in the graph world until Chat GPT came around and then everyone forgot about them. But this is a graph vector. It's a graph that represents not the meaning space of a word in three-dimensional vector space or n-dimensional vector space. It's a vector that represents the shape of the graph around a particular node that lets you then do things like topological similarity and say, here's a high value customer or here's a known fraudulent customer. How is there a match to either for this particular thing coming in? These are all techniques that can be and are being used under the umbrella of GraphRag.
Yeah, like you mentioned, graphrag is a paradigm, so there's many ways to using that. And it's related to the large language model and it's enabled by the more powerful models. But also there's some interesting relationship between that, like you mentioned, the GNN was very popular before LLM. Now most people would do the similar thing with LLM inside.
Or I would say most people haven't remembered or don't yet realize that you can use GNNs in companionship with your LLM to do graph similarity and then inform the LLM with that. I see GNNs as having a role inside of graph rag. It's still pretty early days, so we'll see how that one shows up.
And related to that, actually have also, I know you write or coined a term called a locus of reasoning. Can you tell us more about that?
Yeah, locus of reasoning is like, where does your reasoning happen? And this is an important question in AI is, is the LLM fully responsible for doing all the reasoning in itself that has implications to explainability, implications to understanding your understanding your results and the quality of results and knowing whether your results are, what's the probability of it being correct or incorrect? With graphs, what's interesting is in the case I was mentioning, as an employee, I'm going to ask the LLM, based on my current job, what are the different ways that I can get to this other job that's interesting to me, either because it's in a different location or it's a more senior role in my department or it's because another department that's interesting to me where I might have some prior background. That answer is best gotten by doing a graph walk. In that case, what you want the model to do is take that question, turn it into a graph query, delegate the question to the graph, have the model then take that answer back, convert it back into language, but not mess with the answer. In that case, your locus of reasoning is in the graph. Or you have this in between case where you can say, I'm going to ask my model a question, but then my model is going to reach out to the graph, pull some information back from the graph. Maybe like I'm going to start with a vector, pull back certain things, filter the relation based on relationship. So I'll pull back, fill up and all the information multiple levels out related to, I don't know, insurance policies if I'm in the insurance space, and then feed that back to the model, and then the model has an informed result. In that case, you could say that the reasoning happens in the LLM, but is greatly informed by the graph. So I'd say the locus of reasoning is shared there.
Yeah, exactly. I think you provide such a great mental models about how the reasoning is done. In essentially is a lot of people are talking about reasoning LLM, which reasoning happen in LLM itself. But what you point out is that LLM can do reason only when it has the information needs or it has the help in these. So there is this whole bunch of system on top of LLM. The graph database is one example of that that would enable LLM to reason better. And related to that, I am very curious about your takes about the roadmap of companies like OpenAI and Anthropic on the AGI, because they are really pushing forward that there's a monolithic model that would have the world's knowledge and can do the reasoning inside. It's like all in one box. And they're putting a lot of money on that like stock market five hundred billion here on that. What is the take on those?
That's the end of part 1 of our conversation with Philip. In Part 2, He shares his thoughts on AGI, mindful leadership, and building technology grounded in human judgment. Don't miss it! Subscribe to FounderCoHo if you haven't already.