Which Is Higher for 10 Frequent Infosec Duties?

Which Is Better for 10 Common Infosec Tasks?

In late 2023, I wrote an article evaluating how properly ChatGPT and Google Bard deal with writing safety insurance policies. On condition that ChatGPT 4.0 has been obtainable as a paid model, referred to as ChatGPT Plus, for a while now, and Google not too long ago rebranded Google Bard as Gemini (with Gemini Superior obtainable as paid providing), it is a good time to check how the 2 carry out in a head-to-head of the highest 10 use circumstances for info safety professionals.

Earlier than we leap in, the standard generative synthetic intelligence (AI) caveats apply: Watch out of the info you punch in and bear in mind the output could not all the time be dependable.

1. Producing Diagrams or Idea Flows

Each instruments declare to have the ability to generate diagrams and idea flows. Nevertheless, Gemini admits it might solely generate ASCII diagrams, pointing you to extra skilled instruments if you need one thing higher. I requested each instruments to generate a diagram to elucidate the OAuth authentication move.

Gemini whereas represented in ASCII, does the job and breaks it down into usable classes.

ChatGPT hallucinates badly. At first look, whereas the picture seems skilled, it would not characterize OAuth in any respect. The wording is nonsensical, misspelled, or downright illegible: Authiration and Athoricazt anybody?

2. Explaining Structure Diagrams

Each instruments can ingest diagrams and clarify what is going on on. The outcomes are significantly better than what occurs whenever you ask them to generate diagrams. As enter, I used an instance Internet utility firewall (WAF) structure from Edgenexus.

Google Gemini is significantly better at explaining structure diagrams as a result of it is succinct. ChatGPT will do the job simply advantageous; it is only a tad wordy.

3. Deciphering Exploit Code

A typical safety operations (SecOps) exercise is attempting to determine what a particular malware or exploit code does. I took a current Elasticsearch stack overflow public exploit and fed it into every device to see what it understood. There isn’t any clear winner: Each instruments determine the exploit accurately and clarify the top end result, what every portion of the code does, and the way it works.

4. Deciphering Log Information

SecOps professionals typically want to determine what the heck is occurring in log information. I fed each instruments an example CEF format log file of an tried breach and requested every to elucidate what is going on on. Gemini explains it higher, summarizing properly and even suggesting follow-up steps. It additionally clearly states what occurred (tried entry of /and so on/passwd) proper at first and elaborates on the way it got here to that conclusion. Whereas ChatGPT arrives on the identical conclusion, it’s approach too verbose.

5. Writing Insurance policies and Safety Documentation

I will not elaborate an excessive amount of on this and can as an alternative refer you to my earlier article on this subject. I ran the check once more with Gemini, and the outcomes are in step with Bard’s: Gemini clearly understands and generates higher safety documentation than ChatGPT.

6. Figuring out Susceptible Code

Whereas these instruments weren’t designed for (and should not be used for) figuring out susceptible code, they’ll nonetheless do an ample job. I made a decision to check it by feeding each instruments an insecure direct object reference (IDOR) vulnerability example in Python, which additionally incorporates a SQL injection.

ChatGPT accurately recognized each vulnerabilities and the dearth of authentication. Gemini missed the IDOR however identified the SQL injection and went a step additional to suggest amended code to repair the vulnerability. ChatGPT can even do that, nevertheless it have to be prompted to take action.

7. Writing Scripts and Code

A typical safety operations middle (SOC) exercise is writing  scripts for log parsing or knowledge manipulation. I gave each instruments the next immediate:

“Write me a Python script that extracts all IPv6 addresses from a txt enter file, removes all duplicates, does a lookup to geo-locate and determine the proprietor of the IP, and output the end in a CSV file”

There isn’t any clear winner right here; each instruments produce clear, readable code that works and explains what it does.

8. Analyzing Knowledge and Metrics

I additionally examined whether or not these instruments may assist with analyzing knowledge or safety metrics. Gemini is an enormous loser right here as a result of it would not do it in any respect — it might solely information you thru how do that in Excel and Energy BI. ChatGPT has the benefit by its Knowledge Analyst plug-in, which ingests Excel information to generate any graphs you need. It even suggests visualization varieties, and you’ll modify a graph’s design, together with shade, axes, and labels, by the immediate. 

Example of graph output by ChatGPT's Data Analyst plug-in

9. Writing Consumer Consciousness Messages

Each instruments can even generate emails for safety consciousness campaigns. I gave each the next immediate: “Generate an electronic mail used for a safety consciousness marketing campaign. Be humorous and sarcastic. Remind individuals why they should not click on on random emails from random individuals.”

Gemini wins right here — its electronic mail is temporary, has the fitting tone, and (though humor is subjective) I discovered it barely funnier. ChatGPT nonetheless generates the fitting tone and a superb electronic mail, however I discovered it a tad too lengthy for an consciousness electronic mail. Both approach, each instruments do a fantastic job.

Gemini outputs a user awareness email after being given a prompt

10. Deciphering Compliance Frameworks

When you have a fast query about learn how to implement a compliance framework, these instruments can positively assist. Whilst you could not do that typically, they’re very helpful whenever you want it.

If you happen to’ve ever argued with somebody about what constitutes a “vital” change below PCI-DSS and the way it needs to be utilized, you are not alone. I prompted every device with: 

“Clarify the idea of ‘vital change’ within the context of PCI-DSS. What constitutes a significant change normally? Checklist the precise requirement from the usual as properly”

Gemini has the higher hand: It accurately lists the precise necessities from the usual (corresponding to 6.4.5 and 6.4.6) and learn how to interpret whether or not one thing is a big change. ChatGPT would not point out precisely the place this info seems in the usual. 

Which AI Is Higher, ChatGPT or Gemini?

There you might have it. Relying in your use case, both device generally is a useful ally in boosting productiveness and serving to you in your day-to-day actions within the infosec trenches.

Notify of
Inline Feedbacks
View all comments
Previous Post
ChatGPT Plugins

Third-Celebration ChatGPT Plugins May Result in Account Takeovers

Next Post
Python Remote Administration Tool Via Telegram

Python Distant Administration Instrument By way of Telegram

Related Posts