Log4j C.F.

HomerSimpson98 · 12-14-2021, 09:59 PM

For all my fellow I.T. admins and security geeks out there, can I get a FUCK YOU for the past 7 days? Brutal.

Solecismic · 12-14-2021, 10:13 PM

Quote:

Originally Posted by HomerSimpson98

For all my fellow I.T. admins and security geeks out there, can I get a FUCK YOU for the past 7 days? Brutal.

Is it as bad as it sounds? Is there anything those of us not running a server should do? Any risk that banks or other financial institutions are compromised?

sterlingice · 12-14-2021, 10:22 PM

We haven't run across a whole lot that actually runs it. But I think there's also some measure of we're worried about what we don't know running it.

SI

HomerSimpson98 · 12-14-2021, 10:29 PM

Yessir it is. Apache is a very widely-used piece of software that is used in an untold number of applications, spanning a multitude of business functions. I've been with a Fortune 100 company for over 20 years and I've never seen this level of panic before. We have a few sleepless nights ahead.

HomerSimpson98 · 12-14-2021, 10:30 PM

The bitch of it is that we patched our stuff late last week to paper over the vulnerability. And now another one has been identified.

sterlingice · 12-14-2021, 11:05 PM

Yeah, I saw the new patch just today.

But we use a lot of off the shelf stuff to run much of the enterprise (at least the stuff I'm aware of). Looking off the best list I've seen (log4shell/software at main · NCSC-NL/log4shell · GitHub) most of what we use is listed as "not vulnerable", though with the caveat above of there being vulnerabilities in the software that even devs are not aware of.

There's a lot of big stuff that's just fine. Some of our virtualization stuff has no issues (Citrix, Ivanti) though VMware looks like they're in deep (not surprising, considering how they like to bolt their code together). Client management looks ok - SCCM isn't on there, though JAMF is if you're managing Macs. I could see huge implications if you're a big web shop. And, perusing over the list, I'm glad I'm not a networking person. Plus there are a number of "investigation" lines. What is "impressive" is the breadth of things this impacts that are seemingly unrelated.

I feel like we were scrambling more with Spectre/Meltdown vulnerabilities since it was a hardware issue that affected practically all hardware and had to be mitigated (and not really fixed) with software patching.

But, yeah, this is indeed a mess. So I'm with you on the expletive throwing (that said, there doesn't need to be anything special going on for expletive throwing in an IT shop - saltiness comes with the job... well, or really, the customers)

SI

Galaril · 12-14-2021, 11:07 PM

Quote:

Originally Posted by HomerSimpson98

For all my fellow I.T. admins and security geeks out there, can I get a FUCK YOU for the past 7 days? Brutal.

As the CISO for a large manufacturer I agree. Brutal.Yes long nights the last week and ahead I fear. We have tried to block the IOCs and have seen numerous attempts to exploit this coming out of one nation state .So it is being actively exploited still for sure.

Toddzilla · 12-15-2021, 07:25 AM

For the first time i can ever remember at my company (Capital One), the enterprise has halted all dev work until we are 100% patched.

sovereignstar v2 · 12-15-2021, 09:27 AM

Dealing with this is the only thing I've worked on the entire last week.

Ksyrup · 12-15-2021, 09:34 AM

As a non-IT person (and attorney) who is involved in my company's continuing efforts to come into compliance with various data security requirements that apply to the insurance industry, this shit scares me - especially since I have approximately zero percent understanding of any of the technical aspects, so I'm left to rely on internal staff and the companies we have engaged to help us figure stuff out while active intrusions are out there.

flere-imsaho · 12-15-2021, 09:35 AM

Jesus, I had no idea it was this bad. Glad I left IT a long time ago. Commiserations for those of you stuck in the middle of it.

flere-imsaho · 12-15-2021, 09:43 AM

OK, did some reading and now I understand why it's so bad. Log4j is logging software that's been integrated into a wide variety of popular networking frameworks. So, you could easily deploy a given framework and not even know you had Log4j running, which is probably exacerbated by the fact that logging software is usually a pretty under-the-radar utility.

According to some quotes I read online, the vulnerability in the software basically allows an attacker almost unfettered access to the entire system, which means by the time you identify that you've been compromised, not only do you need to patch the vulnerability, but then you need to go root-and-branch through your system to see what all the attacker might have left behind for their further use.

If your laptop/desktop was compromised in this fashion, the typical recommendation would be to do a complete erase and reinstall from scratch, and even then you might miss something. This is not really an option available to enterprise software.

Good luck, guys. Ooof, my heart goes out to you.

Drake · 12-15-2021, 10:45 PM

Unless I'm missing something, this is one of the (rare) times that I'm glad we're running Windows web servers with no Java components.

NobodyHere · 12-16-2021, 07:37 AM

Quote:

Originally Posted by flere-imsaho

Jesus, I had no idea it was this bad. Glad I left IT a long time ago. Commiserations for those of you stuck in the middle of it.

A person can leave IT?

HomerSimpson98 · 12-16-2021, 08:51 AM

Quote:

Originally Posted by NobodyHere

A person can leave IT?

I thought it was only when they were caught stealing monitors. And even then, its like a 6-strike policy.

sterlingice · 12-16-2021, 11:08 AM

Basements have exits?

SI

12-14-2021, 10:22 PM	#3
sterlingice Hall Of Famer Join Date: Apr 2002 Location: Back in Houston!	We haven't run across a whole lot that actually runs it. But I think there's also some measure of we're worried about what we don't know running it. SI __________________ Houston Hippopotami, III.3: 20th Anniversary Thread - All former HT players are encouraged to check it out! Janos: "Only America could produce an imbecile of your caliber!" Freakazoid: "That's because we make lots of things better than other people!"

12-14-2021, 11:05 PM	#6
sterlingice Hall Of Famer Join Date: Apr 2002 Location: Back in Houston!	Yeah, I saw the new patch just today. But we use a lot of off the shelf stuff to run much of the enterprise (at least the stuff I'm aware of). Looking off the best list I've seen (log4shell/software at main · NCSC-NL/log4shell · GitHub) most of what we use is listed as "not vulnerable", though with the caveat above of there being vulnerabilities in the software that even devs are not aware of. There's a lot of big stuff that's just fine. Some of our virtualization stuff has no issues (Citrix, Ivanti) though VMware looks like they're in deep (not surprising, considering how they like to bolt their code together). Client management looks ok - SCCM isn't on there, though JAMF is if you're managing Macs. I could see huge implications if you're a big web shop. And, perusing over the list, I'm glad I'm not a networking person. Plus there are a number of "investigation" lines. What is "impressive" is the breadth of things this impacts that are seemingly unrelated. I feel like we were scrambling more with Spectre/Meltdown vulnerabilities since it was a hardware issue that affected practically all hardware and had to be mitigated (and not really fixed) with software patching. But, yeah, this is indeed a mess. So I'm with you on the expletive throwing (that said, there doesn't need to be anything special going on for expletive throwing in an IT shop - saltiness comes with the job... well, or really, the customers) SI __________________ Houston Hippopotami, III.3: 20th Anniversary Thread - All former HT players are encouraged to check it out! Janos: "Only America could produce an imbecile of your caliber!" Freakazoid: "That's because we make lots of things better than other people!" Last edited by sterlingice : 12-14-2021 at 11:07 PM.

12-15-2021, 09:34 AM	#10
Ksyrup This guy has posted so much, his fingers are about to fall off. Join Date: Nov 2000 Location: In Absentia	As a non-IT person (and attorney) who is involved in my company's continuing efforts to come into compliance with various data security requirements that apply to the insurance industry, this shit scares me - especially since I have approximately zero percent understanding of any of the technical aspects, so I'm left to rely on internal staff and the companies we have engaged to help us figure stuff out while active intrusions are out there. __________________ M's pitcher Miguel Batista: "Now, I feel like I've had everything. I've talked pitching with Sandy Koufax, had Kenny G play for me. Maybe if I could have an interview with God, then I'd be served. I'd be complete." Last edited by Ksyrup : 12-15-2021 at 09:34 AM.

12-16-2021, 11:08 AM	#16
sterlingice Hall Of Famer Join Date: Apr 2002 Location: Back in Houston!	Basements have exits? SI __________________ Houston Hippopotami, III.3: 20th Anniversary Thread - All former HT players are encouraged to check it out! Janos: "Only America could produce an imbecile of your caliber!" Freakazoid: "That's because we make lots of things better than other people!"

12-14-2021, 10:29 PM	#4
HomerSimpson98 High School Varsity Join Date: Jan 2012 Location: Cowtown, TX	Yessir it is. Apache is a very widely-used piece of software that is used in an untold number of applications, spanning a multitude of business functions. I've been with a Fortune 100 company for over 20 years and I've never seen this level of panic before. We have a few sleepless nights ahead.

12-14-2021, 10:30 PM	#5
HomerSimpson98 High School Varsity Join Date: Jan 2012 Location: Cowtown, TX	The bitch of it is that we patched our stuff late last week to paper over the vulnerability. And now another one has been identified.

12-15-2021, 07:25 AM	#8
Toddzilla Pro Starter Join Date: Jan 2001 Location: Burke, VA	For the first time i can ever remember at my company (Capital One), the enterprise has halted all dev work until we are 100% patched.

12-15-2021, 09:27 AM	#9
sovereignstar v2 hates iowa Join Date: Oct 2010	Dealing with this is the only thing I've worked on the entire last week.

12-15-2021, 09:35 AM	#11
flere-imsaho Coordinator Join Date: Sep 2004 Location: Chicagoland	Jesus, I had no idea it was this bad. Glad I left IT a long time ago. Commiserations for those of you stuck in the middle of it.

12-15-2021, 09:43 AM	#12
flere-imsaho Coordinator Join Date: Sep 2004 Location: Chicagoland	OK, did some reading and now I understand why it's so bad. Log4j is logging software that's been integrated into a wide variety of popular networking frameworks. So, you could easily deploy a given framework and not even know you had Log4j running, which is probably exacerbated by the fact that logging software is usually a pretty under-the-radar utility. According to some quotes I read online, the vulnerability in the software basically allows an attacker almost unfettered access to the entire system, which means by the time you identify that you've been compromised, not only do you need to patch the vulnerability, but then you need to go root-and-branch through your system to see what all the attacker might have left behind for their further use. If your laptop/desktop was compromised in this fashion, the typical recommendation would be to do a complete erase and reinstall from scratch, and even then you might miss something. This is not really an option available to enterprise software. Good luck, guys. Ooof, my heart goes out to you.

12-15-2021, 10:45 PM	#13
Drake assmaster Join Date: Feb 2001 Location: Bloomington, IN	Unless I'm missing something, this is one of the (rare) times that I'm glad we're running Windows web servers with no Java components.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)