Welcome everyone, to One Identity's Virtual Unite Conference. My name is Neil Boyd. I'm the syslog-ng sales specialist for One Identity. My job today is to give you a virtual introduction to log management, and why it is so important in today's high velocity IT environments. Log domain was once the domain of developers and support. It was originally used for application teams and level 3 support Teams for deep dive troubleshooting on applications or IT transactions.
The large well-known correlation engines themselves were actually sold originally to operations teams to streamline IT operations. But as IT security issues became more and more prevalent, the role of the correlation engines really grew. And today you know these applications as SIEMs, or security information event management platforms. Hacking, privacy concerns, information security, all of that has increased the dependence on log data as your SOC's main data source.
That's resulted in an explosion of data requirements across all firms. More infrastructure and endpoints to monitor has meant an explosion in data requirements. The problem is that the quantities of data are massive, and the delivery and ingestion of all of this log data, the problems associated with that, are not trivial. And of course, your costs just keep going up and up. Now, there are standards to help you manage all of this data. Log data is governed by RFC 5424, the syslog protocol.
And it governs such things as standard message format. It governs the relevant terms and definitions. How messages are handled. It has a ranking facility for severity standards. And even allows for vendor specific extensions in a structured way. syslog-ng, which is One Identity's log management platform, follows the RFC 5424 protocol. And the previous 3164, which is now obsolete.
But because nothing is ever easy, many vendors do not actually conform to those standards. And this adds further complexity to your IT Teams efforts to send data in the proper format to your SIEM. Now, many of the SIEM vendors ship their products with their own agents or collectors to ingest data into their platforms. And while that's well and good for those platforms, these agents tend not to integrate well with other platforms.
They're very good at getting data into their own platform, but not so good at getting data into other platforms. And really, this is a core responsibility of good log management. You need a flexible and agnostic log management platform that can ingest from any source, and feed any destination with all of your data properly formatted and ready for further processing. It's how we assure downstream data quality. And in fact, data quality is really what it's all about.
By centrally managing your log data, you can lower the complexity of data sources and network routes. SIEM vendors charge on data consumed or processors required to. Your costs are further impacted by data storage requirements, which may be driven by security or compliance requirements. The data explosion has driven your SOC operation's costs higher and higher, but by adding a log management layer, you can address these issues and lower your costs.
So what are some of the benefits of log management? First is data quality. A log management layer is an abstraction layer between log source producers and log consumers. Because the sources themselves are varied, the logs they send are not all the same. And while the destinations they are heading to also have their own protocols for ingestion, the log management layer will normalize this data, parse it for fast processing downstream, filter off unnecessary logs, and feed the downstream applications in real time with the formats they require.
This ultimately reduces your costs, first by optimizing the SIEM by sending it only the data it requires, and potentially reducing the infrastructure in place to do this. Finally, it adds to the security of the firm by encrypting this data both in motion and at rest. A standard that we see many of our customers already requiring their IT Teams to deliver on.
And that brings us to our part in this story. syslog-ng has been there since the very beginning. Starting out as a project by a Hungarian grad student, he built syslog-ng back in 1998. He and his partners formed Balabit and introduced the open source version of syslog-ng, which was widely accepted around the world and grew to become the de facto industry standard.
By 2007, Premium Edition was introduced. And then in 2008, syslog-ng Store Box, which is an appliance log management device that runs syslog-ng PE at its core, that product was introduced. The other point I would like to add on this slide is that in 2018, Balabit was acquired by One Identity, and that's how I got here.
Something that is really important to point out here, is that this is really a mature market. There are a lot of competitors. syslog-ng, however, has an advantage today that no one has been able to match, and that's in its scalability and performance. The software was written in C. It was designed from the very beginning to scale massively, to be deployed in global disparate environments.
Today, syslog-ng PE sits at the heart of the world's most demanding IT environments in financial services and telecom, manufacturing and government. So now let's look at an example of a high level architecture to give you an idea about what this looks like in your environment. Over here on the left, you can see different data sources. These are all generating logs. And they're sending ultimately to these downstream destinations. Maybe you only have a couple of destinations, such as your SIEM and maybe some Kafka destinations for development, or some databases.
All of these sources, whether they're virtual machines, databases, security devices, network devices, servers, be they Windows or Linux, need to be able to send logs ultimately to their destinations in a fast and readable format that these destinations can consume from. By inserting a log management layer here, which syslog-ng is, you can optimize the amount of data that's going to your SIEM, as well as feed your other destinations in real time as well. The syslog-ng also uses the notion of syslog-ng relays which can be placed close to the hosts themselves.
This gives you some advantage as you can see here in this layer here different transport protocols that the sources use to send their log data to the syslog server. By putting the relays closer to the sources, we can convert for example UDP traffic to TCP, or even send via proprietary log transfer protocol we have called ALTP to guarantee log delivery to the syslog-ng server itself. Ultimately, what happens, you can see the filtering and parsing that goes on either on the relays or on the syslog server or even both, is that this data is formatted into the proper format to deliver the data downstream to its specific destination where it's supposed to go.
So this is also a good opportunity to talk a little bit about licensing. As I mentioned earlier, the SIEMs generally license either on the amount of data that they process and store, or they license on the number of processors required to run all of your data. syslog-ng does not price that way. It's priced based on the number of unique log source hosts that the server will consume from. So a log source host is any unique IP address or hostname device that's going to be sending syslog to the server itself. So into this platform.
This, of course, can scale massively. You could have many multiple servers here. You could have many, many relays scattered around the world and sending this data over the way back to the server. But ultimately, it all depends on how much data the servers themselves are consuming. The licenses that we sell are perpetual.
We don't sell it as a service. We don't have access to your data. We don't have access to the servers. So the only annual fees you'll pay once you've deployed syslog-ng server is on annual support. And this is another way in which firms are able to reduce the costs of their SIEM management layer, their SOC, by reducing the amount of data that ultimately has to be used for security processing that goes to their SIEM.
I want to show you sort of where customers are taking syslog management now. Just picture this as from the previous slide, similar to what you see in terms of on prem data collection. You could have many multiple syslog servers and relays across many multiple data centers or regions, all pushing data back to central data centers for further processing and log collection. And then ultimately off to the cloud.
And so here I've got some examples of Azure, Sentinel, Splunk, Cloud, and QRadar. But it could be really any cloud based SIEM that's collecting data from your on prem devices or applications. The other area that I want to point out is the leveraging of cloud storage to further reduce the costs of this processing. So with these two destinations, both in the cloud, you send your key security events, say to Splunk in this example, and the balance of all the log data can go to, in this example, GCP or elastic.
Doesn't really matter, whatever cloud based system you want to leverage for reduced storage costs or looking at that data later on, this is a very cost effective way of taking advantage of that, since the SIEMs generally charge again on the amount of data they process. And for storage of that data over time. So we see a lot of firms moving to this particular type of architecture. And it's all designed around both having lower cost cloud SIEM management and lower cost cloud storage.
Keep your cost down, we have customers with many petabytes of data now stored in the cloud. And as you now if you're using file storage or on prem storage, that can be extraordinarily expensive. Same as if you're feeling all 100% of your data into your SIEM. Those costs do nothing but become more and more expensive every year. This gives you a more predictable trajectory about where your costs are going long term. And we see a lot of folks leveraging these sort of destinations now.
So that includes my introduction to log management. Here at Virtual Unite, however, you have several opportunities to learn in much greater detail how we actually do these things. We have four other presentations here that demonstrates syslog-ng Premium Edition, and how we optimize your SIEM. We're showing cloud integration to GCP via Google Pub/Sub. We're showing the integration to Azure Sentinel. We're showing Windows Event Collection. And we also have a presentation on the differences between Premium Edition and the syslog-ng open source version. So if you'd like to learn more about how we actually deliver on the benefits of a log management platform, please jump right in and check these out.
Finally, I hope you leave this presentation with an understanding of how adding a log management layer to your enterprise will improve data quality and improve your SIEM performance. Log management will reduce your infrastructure footprint, lower your SOC costs, and improve the IT security of your organization. Please check out the other syslog-ng presentations here at Virtual Unite.
If you have questions or would like to learn more, my contact information will follow. Please reach out. We would love to hear about your issues related to logging, and see how we can help. Again, my name is Neil Boyd. My email address is Neil.Boyd@oneidentity.com. I'm the syslog-ng sales specialist. And we do hope to hear from you, and thank you for attending this session and all of our sessions.
12:49
