Security Posts
The TikTok Hearing Revealed That Congress Is the Problem
The interrogation of CEO Shou Zi Chew highlighted US lawmakers’ own failure to pass privacy legislation.
Categorías: Security Posts
Update: oledump.py Version 0.0.73
A small update to plugin_msi_info to provide extra info on streams.
Indicator ! marks PE and CAB files.
Indicator ? marks files that are not images (and are not marked with !).
The idea is to first inspect streams marked with ! and ?.
The plugin also provides an overview of the files contained inside the CAB file.
oledump_V0_0_73.zip (http)
MD5: 0CAFC87E62E5BC069568B78C1CEE720D
SHA256: CA67FCFA1F4C79668C9ED0C791AFA9D5EEF370AD58DDC542E2204A080A58F9A5
MD5: 0CAFC87E62E5BC069568B78C1CEE720D
SHA256: CA67FCFA1F4C79668C9ED0C791AFA9D5EEF370AD58DDC542E2204A080A58F9A5
Categorías: Security Posts
TikTok Paid for Influencers to Attend the Pro-TikTok Rally in DC
The embattled social media company brought out the checkbook to ensure at least 30 of its biggest assets—creators—were in DC to help fend off critics.
Categorías: Security Posts
Overview of Content Published in February
Content:
Here is an overview of content I published in February:
Blog posts:
SANS ISC Diary entries:
Categorías: Security Posts
Threat Source newsletter (March 23, 2023) — Meta is threatening to ban news sharing in Canada. Good.
Welcome to this week’s edition of the Threat Source newsletter.After asking ChatGPT to write the newsletter for me two weeks ago, I was tempted to have Google’s Bard do the same, but I resisted making this the newsletter’s new gimmick.Instead, I wanted to write about another tech giant — Meta.The company recently doubled down on a threat to remove news links and sharing from its Facebook and Instagram platforms if Canada passes its proposed Online News Act, or bill C-18. The proposed legislation would compel companies like Meta and Google to sign agreements with Canadian news organizations that would pay them each time a user clicks on a news link through one of their platforms (i.e., via a shared link on Facebook or a Google search result).But as the great Tobey Maguire once said in the cinematic classic “Spider-Man:” “I fail to see how that’s my problem.”If Facebook stops users from sharing news links on their pages, it could be a net positive. Facebook users are notoriously the biggest offenders for sharing fake news and misinformation. A May 2020 study published in Nature Human Behavior found that Facebook pointed users to fake news websites during the 2016 presidential election at a higher rate than any other social media platform.A separate study from Harvard found that during the first few months of 2020, the rate of user engagement with fake news to mainstream news stories was 1:3.5, and the International Communications Association found via a study of social media users that, “sharing countermedia content on Facebook is positively associated with ideological extremity and negatively associated with trust in the mainstream news media.”If Instagram, Facebook and other social media sites were to follow along with this with Canada (Google already started quietly removing news links from its search engine last month in protest of the Online News Act), I think it could go a long way toward fighting disinformation. If users can’t get their news through social media, they may be forced to seek out information independently rather than blindly clicking “share” on Great Aunt Betty’s post, which is just a bad parody from the Babylon Bee.I also would be remiss to not discuss the benefits this legislation would possibly have on newsrooms in Canada. As a former journalist, and someone who was worried about being laid off 24/7 in my previous jobs, it’s a financial struggle out there right now for legitimate news organizations. Online advertising isn’t what it once was, so many outlets are being forced to pivot to hard paywalls or rely on clickbait articles that don’t deliver any news. If this presents a new way to fund legitimate journalism, especially if the only financial burden falls on the richest companies in the world, it could go a long way to sustaining newsrooms.Just because something becomes legal in Canada doesn’t mean other countries are going to be adopting the same rules any time soon. But if news sharing does suddenly go away on Facebook in Canada, maybe it will force all of us to think about where we’re really consuming our news from and how we consumed news even just 15 years ago.The one big thingWe’re still reminding people to update their Microsoft Outlook clients as soon as possible after the disclosure of CVE-2023-23397. Attackers have reportedly been exploiting this vulnerability since last year, though a fix is available now through Microsoft. Adversaries could manipulate a targeted system into supplying the user’s Net-NTLMv2 hash to the attacker, which can then be used in NTLM Relay attacks against other systems.Why do I care?Multiple sources, including Microsoft itself, have confirmed that this vulnerability is being used in the wild. Plus, users don’t even have to open the email or any malicious attachments to trigger this vulnerability, the specially crafted email just has to hit the target’s Outlook inbox. This is a high-severity, low-complexity vulnerability everyone should be patching for if they haven’t already.So now what?Microsoft has released a patch that should be applied, but Talos also has several layers of detection and protection available. If, for some reason, your organization cannot apply this patch, Microsoft also provided a few mitigation options, including adding users to the Protected Users Security Group to prevent the use of NTLM as an authentication mechanism as well as blocking port TCP/445 outbound from your network to block the NTLM messages from leaving the network.Top security headlines of the weekThe popular dark web site BreachForums shut down this week after the FBI arrested its main admin. This is the latest in a string of law enforcement wins against cybercrime groups, who also brought down the Hive ransomware gang in January and RaidForums, BreachForums’ predecessor, last year. The site’s administrator, who goes by the username “Pompompurin,” also claimed responsibility for a data breach of the FBI’s email system in November 2021. Cyber criminals commonly used BreachForums to buy and sell stolen databases of information and had been at the center of recent high-profile data breaches, including this month's attack on DC Health Link that led to the theft of sensitive information belonging to several Congressional representatives. (Krebs on Security, Axios)Google’s security research team discovered several zero-day vulnerabilities in certain Samsung chips that leave many Google smartphones and other wearable devices vulnerable. There are four critical flaws that could compromise affected devices “silently and remotely” over the cellular network, according to Google Project Zero’s blog post on the matter. An attacker could exploit those vulnerabilities to “remotely compromise a phone at the baseband level with no user interaction and require only that the attacker know the victim’s phone number.” Google says it was forced to disclose the vulnerabilities without a patch for many of the affected devices because Samsung did not adhere to its 90-day deadline to issue a fix. (TechCrunch, Google Project Zero)TikTok’s CEO was scheduled to appear before a U.S. Congressional committee Thursday to discuss the popular app’s data security and privacy policies as there are renewed calls among the federal government to block the app. Prepared statements from CEO Shou Zi Chew showed that he would tout TikTok’s $1.5 billion investment in storing U.S. users’ information on Oracle servers and allow outside monitors to inspect the company’s source code. U.S. regulators have reportedly threatened to ban TikTok unless the company’s Chinese owners sell their stake, though the actual mechanics of blocking and de-listing the app are more complicated than they seem on the surface. (ABC News, New York Times)Can’t get enough Talos?
SHA 256: 00ab15b194cc1fc8e48e849ca9717c0700ef7ce2265511276f7015d7037d8725
MD5: d47fa115154927113b05bd3c8a308201
Typical Filename: mssqlsrv.exe
Claimed Product: N/A
Detection Name: Trojan.GenericKD.65065311SHA 256: e4973db44081591e9bff5117946defbef6041397e56164f485cf8ec57b1d8934
MD5: 93fefc3e88ffb78abb36365fa5cf857c
Typical Filename: Wextract
Claimed Product: Internet Explorer
Detection Name: PUA.Win.Trojan.Generic::85.lp.ret.sbx.tgSHA 256: de3908adc431d1e66656199063acbb83f2b2bfc4d21f02076fe381bb97afc423
MD5: 954a5fc664c23a7a97e09850accdfe8e
Typical Filename: teams15.exe
Claimed Product: teams15
Detection Name: Gen:Variant.MSILHeracles.59885SHA 256: 280c8c4f08700f0fea08f0e3ca6e96eadccf49c414c56b6a855c945769678e66
MD5: cd1f364e46c6367dd96f8469eb226981
Typical Filename: cd1f364e46c6367dd96f8469eb226981.scr
Claimed Product: N/A
Detection Name: Win.Dropper.Upatre::dkSHA 256: 5616b94f1a40b49096e2f8f78d646891b45c649473a5b67b8beddac46ad398e1
MD5: 3e10a74a7613d1cae4b9749d7ec93515
Typical Filename: IMG001.exe
Claimed Product: N/A
Detection Name: Win.Dropper.Coinminer::1201
- New threat actor wages espionage campaigns across Central Asia and Eastern Europe
- Threat Roundup for March 10 - 17
- Vulnerability Spotlight: Netgear Orbi router vulnerable to arbitrary command execution
- Vulnerability Spotlight: WellinTech ICS platform vulnerable to information disclosure, buffer overflow vulnerabilities
- Talos Takes Ep. #131: Why does the Prometei botnet keep growing?
SHA 256: 00ab15b194cc1fc8e48e849ca9717c0700ef7ce2265511276f7015d7037d8725
MD5: d47fa115154927113b05bd3c8a308201
Typical Filename: mssqlsrv.exe
Claimed Product: N/A
Detection Name: Trojan.GenericKD.65065311SHA 256: e4973db44081591e9bff5117946defbef6041397e56164f485cf8ec57b1d8934
MD5: 93fefc3e88ffb78abb36365fa5cf857c
Typical Filename: Wextract
Claimed Product: Internet Explorer
Detection Name: PUA.Win.Trojan.Generic::85.lp.ret.sbx.tgSHA 256: de3908adc431d1e66656199063acbb83f2b2bfc4d21f02076fe381bb97afc423
MD5: 954a5fc664c23a7a97e09850accdfe8e
Typical Filename: teams15.exe
Claimed Product: teams15
Detection Name: Gen:Variant.MSILHeracles.59885SHA 256: 280c8c4f08700f0fea08f0e3ca6e96eadccf49c414c56b6a855c945769678e66
MD5: cd1f364e46c6367dd96f8469eb226981
Typical Filename: cd1f364e46c6367dd96f8469eb226981.scr
Claimed Product: N/A
Detection Name: Win.Dropper.Upatre::dkSHA 256: 5616b94f1a40b49096e2f8f78d646891b45c649473a5b67b8beddac46ad398e1
MD5: 3e10a74a7613d1cae4b9749d7ec93515
Typical Filename: IMG001.exe
Claimed Product: N/A
Detection Name: Win.Dropper.Coinminer::1201
Categorías: Security Posts
S3 Ep127: When you chop someone out of a photo, but there they are anyway…
Listen now - latest episode. Full transcript inside.
Categorías: Security Posts
Cropping and Redacting Images Safely, (Thu, Mar 23rd)
The recent "acropalypse" vulnerabilities in Android and Windows 11 showed yet again the dangers of relying on image processing tools to redact images [1][2]. While many image formats are still fundamentally "pixel" based, many have gone beyond simple "array of pixel" formats. Added compression, metadata, and other optimization features can make it difficult to remove information from images. This is not a new issue and has been a problem many times [3].
In some cases, image modifications are just appended to the original image file and overlayed as the image is displayed. Or files retain older versions to allow users to "undo" edits. And of course there are "bugs" like what we had with the recent image issues.
Here are some approaches to make image redaction safer. But please use them with caution.
Convert Image Formats
One way to remove "undisplayed" information from images is to convert the image to another format (gif->png, or jpeg->gif). In particular, you may lose some of the details in the image if you convert it to a compressed format. But this may actually help the intent of removing additional information from the image. Converting an image will usually remove metadata (like "EXIF" data) from images or at least reduce it. It will also create a new image based on the last version of the original image and remove edits or prior versions of the image. These additional features usually do not translate between different image formats. It can not hurt to review the final product using a simple text tool to see if you can spot meta data, but the data may not always be apparent.
Take a Screenshot
After your image looks "right", take a screenshot of it. This will likely just copy the "pixel representation" of the current image. Just make sure that you do not have anything sensitive displayed on the screen. Even taking a partial screenshot may not be safe enough.
Take a Photo
Take a photo of the screen (or partial screen). This is probably the safest way to remove any information from the original file. But you may add new metadata by taking the image. Also, be aware of reflections and other unintended content included in the photo.
Camera artifacts like lens distortions can theoretically be used to identify the particular camera being used. Reducing the image's resolution may help reduce the probability of this happening.
Remove Metadata
Most images include some form of metadata, for example, EXIF data. There are numerous tools to review and remove or modify the metadata. Some of the data may be necessary to properly display the image. But other data, like camera GPS and other sensor data, should be removed. You may also find data identifying the camera (even serial numbers) that you should remove.
Summary
It is hard to redact images properly. In the end: Try to figure out if it is worth the risk of posting the image. If it is a minor detail you redact, the risk may be acceptable. But if revealing redacted information may get you arrested or fired: Think twice before posting the image.
[1] https://acropalypse.app
[2] https://twitter.com/sjmurdoch/status/1638623990817103888
[3] https://www.wired.com/story/redact-pdf-online-privacy/ ---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter| (c) SANS Internet Storm Center. https://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.
[2] https://twitter.com/sjmurdoch/status/1638623990817103888
[3] https://www.wired.com/story/redact-pdf-online-privacy/ ---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter| (c) SANS Internet Storm Center. https://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.
Categorías: Security Posts
Fighting the Good Fight: Life inside the Talos Ukraine Task Unit
As we spoke about in the new ThreatWise TV documentary, “People Matter: A look back on how Cisco Talos has been supporting Ukraine,” war isn’t something that often appears in an organization’s business continuity or disaster recovery plans.In the months leading up to Russia’s invasion of Ukraine, Cisco and Talos did everything we could to support our friends, partners and colleagues, who were facing a reality unlike anything that can be found in any technical training manual, SOP or SLA.Once the invasion began, there was an influx of people across Cisco and Talos who wanted to help. That led to the development of an internal Ukraine task unit, which has become a prototype for how we can respond to future global events that are likely to have significant, ongoing cyber implications.We also deployed and managed Cisco Secure products within a variety of Ukrainian organizations, and refocused parts of our workforce to monitor and detect threats against critical infrastructure. Much of this work continues today as part of an ongoing, comprehensive wartime effort to protect the people of Ukraine and enhance the resilience of Ukrainian organizations.Many people have asked about our task unit, and what we do on a day-to-day basis to help organizations in Ukraine detect and respond to attacks against their critical infrastructure.As you can probably imagine, there isn’t a typical day.One of the key outcomes of the task unit, which has been wonderful to witness, is that people without a technical threat hunting background can add a great deal to our efforts. The power in diversity of thought and experience is explicit in our efforts to support Ukraine.We decided to encapsulate this difficult, but important, work in the form of a graphic novel, which explores some of the themes we touched on in the documentary. Read it below or click here.Further resources
- Learn about the trends and threats Cisco Talos observed from monitoring Ukraine critical infrastructure.
- For the latest on the cybersecurity situation in Ukraine, visit the Talos hub page.
Categorías: Security Posts
Senderbase.org redirects to end in April
As of April 20, 2023, we are decommissioning SenderBase.org and any attempts to visit that web page will fail.Talos Intelligence’s website (TalosIntelligence.com) has served as the replacement for SenderBase.org for many years, with TalosIntelligence.com providing the same information as SenderBase.org once did. Since that time, visitors to SenderBase.org have been automatically redirected to TalosIntelligence.com, and the redirect from SenderBase.org is finally being removed on April 20, 2023. After that, attempts to visit SenderBase.org will fail.Any users still utilizing bookmarks or links pointing to Senderbase.org should update these to ensure they still work appropriately.Thank you for assisting us in this process.
Categorías: Security Posts
Understanding Managed Detection and Response – and what to look for in an MDR solution
Why your organization should consider an MDR solution and five key things to look for in a service offering
The post Understanding Managed Detection and Response – and what to look for in an MDR solution appeared first on WeLiveSecurity
Categorías: Security Posts
BlackGuard stealer extends its capabilities in new variant
AT&T Alien Labs researchers have discovered a new variant of BlackGuard stealer in the wild, infecting using spear phishing attacks. The malware evolved since its previous variant and now arrives with new capabilities.
Key takeaways:
The malware added persistence to survive system reboot by adding itself under the “Run” registry key. (Figure 12) Figure 12. Setting registry persistence. Documents - stealth activity The malware searches and sends to its command and control all documents end with extensions “.txt”, “.config”, “.docx”, “.doc”, “.rdp” in the user folders (including sub directories): “Desktop”, “My Documents”, UserProfile folder. Detection methods The following associated detection methods are in use by Alien Labs. They can be used by readers to tune or deploy detections in their own environments or for aiding additional research. SURICATA IDS SIGNATURES 2035716: ET TROJAN BlackGuard_v2 Data Exfiltration Observed 2035398: ET TROJAN MSIL/BlackGuard Stealer Exfil Activity Associated indicators (IOCs) The following technical indicators are associated with the reported intelligence. A list of indicators is also available in the OTX Pulse. Please note, the pulse may include other activities related but out of the scope of the report. TYPE INDICATOR DESCRIPTION IP ADDRESS http://23[.]83.114.131 Malware command & control SHA256 88e9780ce5cac572013aebdd99d154fa0b61db12faffeff6f29f9d2800c915b3 Malware hash Mapped to MITRE ATT&CK The findings of this report are mapped to the following MITRE ATT&CK Matrix techniques:
- BlackGuard steals user sensitive information from a wide range of applications and browsers.
- The malware can hijack crypto wallets copied to clipboard.
- The new variant is trying to propagate through removable media and shared devices.
The malware added persistence to survive system reboot by adding itself under the “Run” registry key. (Figure 12) Figure 12. Setting registry persistence. Documents - stealth activity The malware searches and sends to its command and control all documents end with extensions “.txt”, “.config”, “.docx”, “.doc”, “.rdp” in the user folders (including sub directories): “Desktop”, “My Documents”, UserProfile folder. Detection methods The following associated detection methods are in use by Alien Labs. They can be used by readers to tune or deploy detections in their own environments or for aiding additional research. SURICATA IDS SIGNATURES 2035716: ET TROJAN BlackGuard_v2 Data Exfiltration Observed 2035398: ET TROJAN MSIL/BlackGuard Stealer Exfil Activity Associated indicators (IOCs) The following technical indicators are associated with the reported intelligence. A list of indicators is also available in the OTX Pulse. Please note, the pulse may include other activities related but out of the scope of the report. TYPE INDICATOR DESCRIPTION IP ADDRESS http://23[.]83.114.131 Malware command & control SHA256 88e9780ce5cac572013aebdd99d154fa0b61db12faffeff6f29f9d2800c915b3 Malware hash Mapped to MITRE ATT&CK The findings of this report are mapped to the following MITRE ATT&CK Matrix techniques:
- TA0001: Initial Access
- T1091: Replication Through Removable Media
- TA0002: Execution
- T1106: Native API
- T1047: Windows Management Instrumentation
- TA0003: Persistence
- T1547.001: Registry Run Keys / Startup Folder
- TA0005: Defense Evasion
- T1027: Obfuscated Files or Information
- TA0006: Credential Access
- T1003: OS Credential Dumping
- T1539: Steal Web Session Cookie
- T1528: Steal Application Access Token
- T1552: Unsecured Credentials
- .001: Credentials In Files
- .002: Credentials In Files
- TA0007: Discovery
- T1010: Application Window Discovery
- T1622: Debugger Evasion
- T1083: File and Directory Discovery
- T1057: Process Discovery
- T1012: Query Registry
- T1082: System Information Discovery
- T1497: Virtualization/Sandbox Evasion
- TA0008: Lateral Movement
- T1091: Replication Through Removable Media
- TA0009: Collection
- T1115: Clipboard Data
- T1213: Data from Information Repositories
- T1005: Data from Local System
- TA0011: Command and Control
- T1071: Application Layer Protocol
- T1105: Ingress Tool Transfer
- TA0010: Exfiltration
- T1020: Automated Exfiltration
Categorías: Security Posts
Hack-én: La CON de hacking del 5 a 7 de Mayo en Linares (Jaén)
Los días 5, 6 y 7 del mes de Mayo tenemos una cita con la ciberseguridad y el hacking en la CON Hack-én, una conferencia de tres días, con talleres, y ponencias, que tendrá lugar en Linares, (Jaen) y que será el foco ese fin de semana para formarse en nuestra disciplina profesional. La conferencia comienza el viernes al medio día con una serie de ponencias, continúa el sábado con track doble de conferencias y talleres, y termina el domingo de nuevo con conferencias, así que es un campo de aprendizaje acelerado para profesionalizarse en ciberseguridad.
Figura 1: Hack-én: La CON de hacking del 5 a 7 de Mayo en Linares (Jaén)
Entre los ponentes, pues tienes a muchos de los grandes de este país, que es el plato fuerte de este evento de tres días. Entre ellos, Carlos Seisdedos que además de un crack es uno de los grandes expertos en ciberinteligencia. Su libro de Open Source Intelligence se ha convertido en una referencia para todos los que trabajamos en este campo.
Figura 2: Tres días de hacking en la Hack-én
También está Pablo González, que ha formado a una generación de pentesters con sus charlas, cursos, y los libros de Metasploit para Pentesters Gold Edition, Pentesting con PowerShell, Pentesting con Kali Linux, Ethical Linux, Hacking con Metasploit: Advanced Pentesting, Hacking Windows, o Empire: Hacking avanzado en el Red Team.
Figura 3: Ponentes de Hack-én
En la lista de ponentes, que es enorme, encontrarás también a Sergio de los Santos, autor de Máxima Seguridad en Windows y un libro que no debes dejar de estudiar ya que explica cómo las bandas organizadas del cibercrimen están diseñando sus plataformas de Malware Moderno. También están el gran Pablo San Emeterio, José Navarro o Rafael López, todos con amplia experiencia en nuestro sector.
Figura 4: Ponentes de Hack-én
En la lista de ponentes, vas a encontrar a Adrián Ramírez, Jorge Escabias, el gran (que además es alto) Eduardo Sánchez Toril que ha mojado España con sus "Hack & Beers", o al incombustible Daniel Echevarry "Adastra", que además es el autor de los libros de Python para Pentesters 2ª Edición, Hacking con Python o DeepWeb.
Figura 5: Contacta con Hack-én CON en MyPublicInbox
Si quieres colaborar con la organización, o participar de alguna manera, puedes ponerte en contacto con ellos a través de su buzón de MyPublicInbox, y si quieres asistir, puedes comprar tres tipos de entradas para esta edición de Hack-én.
Figura 6: Comprar entrada para Hack-én
Además puedes conseguir un descuento en el precio de las entradas a través de Tempos de MyPublicInbox, así que no tienes excusa para no participar si te gusta el contenido, ya que apoyarás que se sigan realizando este tipo de eventos en la zona.
Figura 7: Consigue un 10% de descuento con Tempos de MyPublicInbox
Para conocer más sobre el evento, les he hecho una entrevista a los organizadores, para que nos cuenten un poco más de todo lo que hay en esta Hack-en de Linares (Jaen), este fin de semana de Mayo. Aquí va.
1. ¿Cómo nace Hack-én, quién está detrás de esta CON?
La idea de Hack-én nació de la necesidad de crear un evento que permitiera reunir a expertos, profesionales y entusiastas de la ciberseguridad en Jaén. El objetivo principal de Hack-én es fomentar la colaboración y el intercambio de conocimientos entre los participantes, así como concienciar sobre la importancia de la ciberseguridad en la actualidad. Hack-én es organizado por un grupo de profesionales del sector de la ciberseguridad que provienen de la provincia de Jaén. Especialistas en diferentes ámbitos tales como: Pentesting & RedTeam, Forense, BlueTeam, Vulnerability Management & Security Architecture, sSDLC, Governance, Risk & Compliance, etcétera.
Figura 8: Contactar con Nicomda en MyPublicInbox
Muchos desde jóvenes, otros no tanto, pero todos relacionados con el sector y apasionados por este mundo. ¡A alguno incluso lo conocéis desde los tiempos de Informática 64! Siendo más específicos, "los de Hack-én" somos: Antonio Cortés, Pablo Cueto, Luís Jesús Montes, Nicolás Moral, Jaime Solás, Álvaro Solás, David Padilla y Victor Pérez.
2. ¿Qué tenéis preparado para esta edición de Hack-én, quiénes son los ponentes que vendrán?
Tenemos preparado un track de formación con ponencias y una sala para talleres que se impartirán de forma simultánea en el Campus Científico Tecnológico de Linares, que pertenece a la Universidad de Jaén, del que varios de los organizadores son egresados. Estamos bastante orgullosos de los ponentes que nos acompañarán en esta primera edición.
Figura 9: Contactar con Carlos Seisdedos, ponente en Hack-én
Tendremos con nosotros a: Carlos Seisdedos, Pablo González, Pablo San Emeterio, Elisa García, Kino Makino (Joaquín Molina), Sergio de los Santos, Omar Jesús Orta, José Navarro, Rafael López, Eduardo Sánchez Toril, Adrián Ramirez, Jorge Escabias... y algunas sorpresas más que están por anunciarse.
3. Además de los tracks de formación ¿qué más actividades tenéis preparadas?
Se realizarán sorteos, un CTF, algún concurso, ¡y algunos retos para hackers! Tenéis una sala de Chat en MyPublicInbox, por si quieres proponer alguna idea, debatir un tema, o compartir algo de lo que deseas de esta CON.
Figura 10: Sala de Chat de Hack-én en MyPublicInbox. Apúntatecon este enlace: https://mypublicinbox.com/chatinvite/vQBQd
4. ¿Qué son las Hack-én Coins?
Es un nuevo proyecto de NFT que se desarrolla desde el equipo de Hack-én... ¡Es broma! Las Hack-én Coins son las monedas virtuales creadas para el congreso, que podrán ser canjeadas por diferentes premios durante el desarrollo del mismo.
Podrán conseguirse al adquirir la entrada, al asistir a ponencias o talleres, participando en el CTF o en otras actividades del congreso. Durante la acreditación, se les dará a los asistentes un lanyard con un QR único. Este QR les permitirá ver cuantas Hack-én Coins tienen y en qué pueden canjearse.
5. ¿Qué van a llevarse los asistentes al evento como aprendizaje?
Los asistentes podrán disfrutar de ponencias y talleres impartidos por algunos de los profesionales más relevantes del sector, teniendo además la oportunidad de interactuar con personas relacionadas con el mundo ciber, lo que les permitirá intercambiar conocimientos, compartir experiencias y establecer contactos valiosos para su desarrollo profesional y empresarial en este campo. ¡Será una experiencia muy enriquecedora para todos!
¡Saludos Malignos!
Autor: Chema Alonso (Contactar con Chema Alonso)
Sigue Un informático en el lado del mal RSS 0xWord
- Contacta con Chema Alonso en MyPublicInbox.com
Figura 1: Hack-én: La CON de hacking del 5 a 7 de Mayo en Linares (Jaén)
Entre los ponentes, pues tienes a muchos de los grandes de este país, que es el plato fuerte de este evento de tres días. Entre ellos, Carlos Seisdedos que además de un crack es uno de los grandes expertos en ciberinteligencia. Su libro de Open Source Intelligence se ha convertido en una referencia para todos los que trabajamos en este campo.
Figura 2: Tres días de hacking en la Hack-én
También está Pablo González, que ha formado a una generación de pentesters con sus charlas, cursos, y los libros de Metasploit para Pentesters Gold Edition, Pentesting con PowerShell, Pentesting con Kali Linux, Ethical Linux, Hacking con Metasploit: Advanced Pentesting, Hacking Windows, o Empire: Hacking avanzado en el Red Team.
Figura 3: Ponentes de Hack-én
En la lista de ponentes, que es enorme, encontrarás también a Sergio de los Santos, autor de Máxima Seguridad en Windows y un libro que no debes dejar de estudiar ya que explica cómo las bandas organizadas del cibercrimen están diseñando sus plataformas de Malware Moderno. También están el gran Pablo San Emeterio, José Navarro o Rafael López, todos con amplia experiencia en nuestro sector.
Figura 4: Ponentes de Hack-én
En la lista de ponentes, vas a encontrar a Adrián Ramírez, Jorge Escabias, el gran (que además es alto) Eduardo Sánchez Toril que ha mojado España con sus "Hack & Beers", o al incombustible Daniel Echevarry "Adastra", que además es el autor de los libros de Python para Pentesters 2ª Edición, Hacking con Python o DeepWeb.
Figura 5: Contacta con Hack-én CON en MyPublicInbox
Si quieres colaborar con la organización, o participar de alguna manera, puedes ponerte en contacto con ellos a través de su buzón de MyPublicInbox, y si quieres asistir, puedes comprar tres tipos de entradas para esta edición de Hack-én.
Figura 6: Comprar entrada para Hack-én
Además puedes conseguir un descuento en el precio de las entradas a través de Tempos de MyPublicInbox, así que no tienes excusa para no participar si te gusta el contenido, ya que apoyarás que se sigan realizando este tipo de eventos en la zona.
Figura 7: Consigue un 10% de descuento con Tempos de MyPublicInbox
Para conocer más sobre el evento, les he hecho una entrevista a los organizadores, para que nos cuenten un poco más de todo lo que hay en esta Hack-en de Linares (Jaen), este fin de semana de Mayo. Aquí va.
1. ¿Cómo nace Hack-én, quién está detrás de esta CON?
La idea de Hack-én nació de la necesidad de crear un evento que permitiera reunir a expertos, profesionales y entusiastas de la ciberseguridad en Jaén. El objetivo principal de Hack-én es fomentar la colaboración y el intercambio de conocimientos entre los participantes, así como concienciar sobre la importancia de la ciberseguridad en la actualidad. Hack-én es organizado por un grupo de profesionales del sector de la ciberseguridad que provienen de la provincia de Jaén. Especialistas en diferentes ámbitos tales como: Pentesting & RedTeam, Forense, BlueTeam, Vulnerability Management & Security Architecture, sSDLC, Governance, Risk & Compliance, etcétera.
Figura 8: Contactar con Nicomda en MyPublicInbox
Muchos desde jóvenes, otros no tanto, pero todos relacionados con el sector y apasionados por este mundo. ¡A alguno incluso lo conocéis desde los tiempos de Informática 64! Siendo más específicos, "los de Hack-én" somos: Antonio Cortés, Pablo Cueto, Luís Jesús Montes, Nicolás Moral, Jaime Solás, Álvaro Solás, David Padilla y Victor Pérez.
2. ¿Qué tenéis preparado para esta edición de Hack-én, quiénes son los ponentes que vendrán?
Tenemos preparado un track de formación con ponencias y una sala para talleres que se impartirán de forma simultánea en el Campus Científico Tecnológico de Linares, que pertenece a la Universidad de Jaén, del que varios de los organizadores son egresados. Estamos bastante orgullosos de los ponentes que nos acompañarán en esta primera edición.
Figura 9: Contactar con Carlos Seisdedos, ponente en Hack-én
Tendremos con nosotros a: Carlos Seisdedos, Pablo González, Pablo San Emeterio, Elisa García, Kino Makino (Joaquín Molina), Sergio de los Santos, Omar Jesús Orta, José Navarro, Rafael López, Eduardo Sánchez Toril, Adrián Ramirez, Jorge Escabias... y algunas sorpresas más que están por anunciarse.
3. Además de los tracks de formación ¿qué más actividades tenéis preparadas?
Se realizarán sorteos, un CTF, algún concurso, ¡y algunos retos para hackers! Tenéis una sala de Chat en MyPublicInbox, por si quieres proponer alguna idea, debatir un tema, o compartir algo de lo que deseas de esta CON.
Figura 10: Sala de Chat de Hack-én en MyPublicInbox. Apúntatecon este enlace: https://mypublicinbox.com/chatinvite/vQBQd
4. ¿Qué son las Hack-én Coins?
Es un nuevo proyecto de NFT que se desarrolla desde el equipo de Hack-én... ¡Es broma! Las Hack-én Coins son las monedas virtuales creadas para el congreso, que podrán ser canjeadas por diferentes premios durante el desarrollo del mismo.
Podrán conseguirse al adquirir la entrada, al asistir a ponencias o talleres, participando en el CTF o en otras actividades del congreso. Durante la acreditación, se les dará a los asistentes un lanyard con un QR único. Este QR les permitirá ver cuantas Hack-én Coins tienen y en qué pueden canjearse.
5. ¿Qué van a llevarse los asistentes al evento como aprendizaje?
Los asistentes podrán disfrutar de ponencias y talleres impartidos por algunos de los profesionales más relevantes del sector, teniendo además la oportunidad de interactuar con personas relacionadas con el mundo ciber, lo que les permitirá intercambiar conocimientos, compartir experiencias y establecer contactos valiosos para su desarrollo profesional y empresarial en este campo. ¡Será una experiencia muy enriquecedora para todos!
¡Saludos Malignos!
Autor: Chema Alonso (Contactar con Chema Alonso)
Sigue Un informático en el lado del mal RSS 0xWord
- Contacta con Chema Alonso en MyPublicInbox.com
Categorías: Security Posts
Windows 11 also vulnerable to “aCropalypse” image data leakage
Turns out that the Windows 11 Snipping Tool has the same "aCropalypse" data leakage bug as Pixel phones. Here's how to work around the problem...
Categorías: Security Posts
Can you pass The Rekt Test?
Audits from Trail of Bits give organizations ways to fix their current issues and provide them with a roadmap to prevent the same security problems from occurring in the future. However, product teams don’t always implement the recommendations. This may be due to a lack of resources, a lack of institutional motivation, or several other internal reasons. Even when action is taken, progress can be difficult to measure.
We now have a way organizations can measure that progress. The Rekt Test provides a framework for an informed conversation that, if moderated by experts, can allow an organization to get a fuller picture of the risk presented by a blockchain project. Modeled after the The Joel Test, an organization should try to answer ‘Yes’ for as many of the below questions as possible before going public with a new blockchain venture. The more an organization can answer ‘Yes’ to these questions, the more stable its security foundation should be.
The Rekt Test
1. Do you have all actors, their roles, and privileges documented? It’s crucial to understand who is operating in your environment — users, developers, miners, validators, and other stakeholders — and the breakdown of their unique responsibilities, tasks, permissions and other duties that may be performed. This should be documented in a clear and concise manner, so an organization can easily identify when a security incident is happening. 2. Do you employ a multi-factor human and physical key management system? The keys to your wallets are always going to have a target on them. For protection, we recommend implementing a two-factor authentication system for users, using both physical and digital backups, and applying a multi-signature system for transactions. For multi-signatures, we suggest a consensus requirement for a sensitive action, like spending funds out of a wallet. This is to mitigate the risks of any singular party’s judgment overruling others, any singular party’s mistake causing a failure, and any singular credential compromise causing a failure. 3. Do you have a written and tested incident response plan? While security in the blockchain space is different from traditional enterprises, an incident response plan is a proven practice that will enable your organization to be resilient in the face of a security incident. The plan should include steps to identify, contain, and remediate the incident, through both automated and manual procedures. An organization should have all team members familiar with the plan, including both internal and external communication channels. This plan should be regularly tested to ensure that it is up-to-date and effective, especially given how quick the blockchain security world can change. As an example of what this traditionally looks like, Carnegie Mellon University has posted their plan online. 4. Do you perform identity verification and background checks on all employees? This is where your human resource officers can help with your security posture. While organizations should be conducting the standard checks — criminal background, employment history, credit check, reference check — a social media background check can unearth behavior that could be problematic from a security perspective. A little OSINT could go a long way. 5. Do you have a team member that has security defined in their role? Answering “yes” to this isn’t as simple as hiring a CISO. (Although that’s better than nothing!) Whether it’s developing something new or updating something that already exists, security cannot be a side gig on someone’s job responsibilities. There needs to be a dedicated person in the organization that can take the lead on making sure a product or service is as secure as possible. This person should also be deeply involved in getting an organization to “yes” on other questions in this list, particularly incident response plans and using the best tools currently on the market. 6. Do you require hardware security keys for production systems? Let’s not mince words: if your internal credentials are leaked or stolen, the fallout will be harsh. Using SMS codes for two-factor authentication is not good enough for the credentials guarding your code. There are several different options that support a wide range of authentication protocols, which will make your business-critical code repositories highly secure and resilient to breach attempts. YubiKey and Google Titan are two good choices for hardware keys. 7. Do you use the best automated tools for discovering security issues in your code? If organizations are going to be constantly testing their security, they are going to need tools that can keep up with the industry’s best practices. Trail of Bits has a wide suite of tools and documentation that will allow you to stay current in your security posture:
For those unfamiliar, an invariant is a condition or value that remains unchanged throughout a program’s execution. It is used to ensure that the program behaves as expected and that the results are consistent. As you develop new features, your organization should be testing to see if the new code plays nice with these invariants. These could be tied to things like cryptographic hash functions, consensus mechanisms, transaction validation, transaction privacy or other network security functions. By continually testing, you are constantly given the opportunity to find issues in your system before an attacker does. 9. Do you undergo external audits and maintain a vulnerability disclosure or bug bounty program? Getting an external audit will positively compound your security posture, both internally and externally. An audit will allow experts with a fresh perspective to search for and fix vulnerabilities that an internal security practice may not be aware of or have the expertise to uncover. Whether the audit falls on the actual smart contract or the components supporting the contract, it will also provide assurances to the public that your organization is taking the necessary steps to independently verify that your security is as good as it can possibly be. A vulnerability disclosure or bug bounty program is an additional way to enhance your security posture. Leading blockchain and web3 companies have set these programs up as a way to tap into the public’s security expertise and address potential issues before they become a problem. Some examples are:
- Do you have all actors, their roles, and privileges documented?
- Do you employ a multi-factor human and physical key management system?
- Do you have a written and tested incident response plan?
- Do you perform identity verification and background checks on all employees?
- Do you have a team member that has security defined in their role?
- Do you require hardware security keys for production systems?
- Do you use the best automated tools for discovering security issues in your code?
- Do you define key invariants for your system and test them on every commit?
- Do you undergo external audits and maintain a vulnerability disclosure or bug bounty program?
- Do you keep documentation of all the external services, contracts, and oracles you rely on?
- Do you document the best ways to attack your own system?
- Do you identify and address potential avenues for users who abuse your system?
1. Do you have all actors, their roles, and privileges documented? It’s crucial to understand who is operating in your environment — users, developers, miners, validators, and other stakeholders — and the breakdown of their unique responsibilities, tasks, permissions and other duties that may be performed. This should be documented in a clear and concise manner, so an organization can easily identify when a security incident is happening. 2. Do you employ a multi-factor human and physical key management system? The keys to your wallets are always going to have a target on them. For protection, we recommend implementing a two-factor authentication system for users, using both physical and digital backups, and applying a multi-signature system for transactions. For multi-signatures, we suggest a consensus requirement for a sensitive action, like spending funds out of a wallet. This is to mitigate the risks of any singular party’s judgment overruling others, any singular party’s mistake causing a failure, and any singular credential compromise causing a failure. 3. Do you have a written and tested incident response plan? While security in the blockchain space is different from traditional enterprises, an incident response plan is a proven practice that will enable your organization to be resilient in the face of a security incident. The plan should include steps to identify, contain, and remediate the incident, through both automated and manual procedures. An organization should have all team members familiar with the plan, including both internal and external communication channels. This plan should be regularly tested to ensure that it is up-to-date and effective, especially given how quick the blockchain security world can change. As an example of what this traditionally looks like, Carnegie Mellon University has posted their plan online. 4. Do you perform identity verification and background checks on all employees? This is where your human resource officers can help with your security posture. While organizations should be conducting the standard checks — criminal background, employment history, credit check, reference check — a social media background check can unearth behavior that could be problematic from a security perspective. A little OSINT could go a long way. 5. Do you have a team member that has security defined in their role? Answering “yes” to this isn’t as simple as hiring a CISO. (Although that’s better than nothing!) Whether it’s developing something new or updating something that already exists, security cannot be a side gig on someone’s job responsibilities. There needs to be a dedicated person in the organization that can take the lead on making sure a product or service is as secure as possible. This person should also be deeply involved in getting an organization to “yes” on other questions in this list, particularly incident response plans and using the best tools currently on the market. 6. Do you require hardware security keys for production systems? Let’s not mince words: if your internal credentials are leaked or stolen, the fallout will be harsh. Using SMS codes for two-factor authentication is not good enough for the credentials guarding your code. There are several different options that support a wide range of authentication protocols, which will make your business-critical code repositories highly secure and resilient to breach attempts. YubiKey and Google Titan are two good choices for hardware keys. 7. Do you use the best automated tools for discovering security issues in your code? If organizations are going to be constantly testing their security, they are going to need tools that can keep up with the industry’s best practices. Trail of Bits has a wide suite of tools and documentation that will allow you to stay current in your security posture:
- Echidna, a smart contract fuzzer
- Slither, a static analyzer for solidity
- Circomspect, a static analyzer and linter for the Circom programming language
- Amarna, a static analyzer and linter for the Cairo programming language
- Building Secure Contracts, guidelines and best practices to write secure smart contracts
- ZKDocs, comprehensive, detailed, and interactive documentation on zero-knowledge proof systems and related primitives.
For those unfamiliar, an invariant is a condition or value that remains unchanged throughout a program’s execution. It is used to ensure that the program behaves as expected and that the results are consistent. As you develop new features, your organization should be testing to see if the new code plays nice with these invariants. These could be tied to things like cryptographic hash functions, consensus mechanisms, transaction validation, transaction privacy or other network security functions. By continually testing, you are constantly given the opportunity to find issues in your system before an attacker does. 9. Do you undergo external audits and maintain a vulnerability disclosure or bug bounty program? Getting an external audit will positively compound your security posture, both internally and externally. An audit will allow experts with a fresh perspective to search for and fix vulnerabilities that an internal security practice may not be aware of or have the expertise to uncover. Whether the audit falls on the actual smart contract or the components supporting the contract, it will also provide assurances to the public that your organization is taking the necessary steps to independently verify that your security is as good as it can possibly be. A vulnerability disclosure or bug bounty program is an additional way to enhance your security posture. Leading blockchain and web3 companies have set these programs up as a way to tap into the public’s security expertise and address potential issues before they become a problem. Some examples are:
- Uniswap has a bug bounty program that incentivizes responsible bug disclosure and offers rewards up to 2,250,000 USDC for bugs.
- Primitive has a program with ImmuneFi that rewards up to $1 million for vulnerabilities that impact loss of user funds or governance funds.
- MakerDAO has a $10 million program with ImmuneFi that has a large library of smart contract code for researchers to look over.
- Gemini customers were the target of phishing campaigns that the result of an incident at a third-party vendor
- Cryptocurrency hardware wallet provider Trezor saw its users targeted with phishing attempts after a security incident at MailChimp.
- Tens of millions in worth have been taken in incidents where attackers targeted data oracles.
- Implement a KYC process to ensure user identity
- Consider a reputation system to track user behavior
- Establish escrow accounts to ensure funds are held securely in the event of a security incident
- Monitor relevant chains for suspicious activity and shut down users in the event of bad behavior
Categorías: Security Posts
Codex (and GPT-4) can’t beat humans on smart contract audits
By Artem Dinaburg, Chief Technology Officer; Josselin Feist, Principal Engineer; and Riccardo Schirone, Security Engineer
Is artificial intelligence (AI) capable of powering software security audits? Over the last four months, we piloted a project called Toucan to find out. Toucan was intended to integrate OpenAI’s Codex into our Solidity auditing workflow. This experiment went far beyond writing “where is the bug?” in a prompt and expecting sound and complete results.
Our multi-functional team, consisting of auditors, developers, and machine learning (ML) experts, put serious work into prompt engineering and developed a custom prompting framework that worked around some frustrations and limitations of current large language model (LLM) tooling, such as working with incorrect and inconsistent results, handling rate limits, and creating complex, templated chains of prompts. At every step, we evaluated how effective Toucan was and whether it would make our auditors more productive or slow them down with false positives.
The technology is not yet ready for security audits for three main reasons:
Figure 1: True positives (green) and false positives (red) found by Slither, Toucan, and both on some simple re-entrancy tests. The Toucan results are not encouraging. We tried looking at the true/false positive sets of Slither and Toucan, and found that each tool had a different set of false positives/false negatives, with some overlap (Figure 1). Codex was not able to effectively reduce the false positive rate from a prototype Slither integer overflow detector. Overall, we noticed a tendency to reply affirmatively to our questions, increasing the number of positives discovered by Toucan. Codex can perform basic static analysis tasks, but the rate of failure is too high to be useful and too difficult to characterize. This capability to perform successful analysis, even on short program fragments, is very impressive and should not be discounted! For languages that Codex understands but for which no suitable tooling exists, this capability could be extremely valuable—after all, some analysis could be much better than nothing. But the benchmark for Solidity is not nothing; we already have existing static analysis tooling that works very well. How we framed our framework During Toucan’s development, we created a custom prompting framework, a web-based front end, and rudimentary debugging and testing tools to evaluate prompts and to aid in unit and integration tests. The most important of these was the prompting framework. Prompting framework If we were making Toucan today, we’d probably just use LangChain. But at the time, LangChain did not have the features we needed. Frustratingly, neither OpenAI nor Microsoft offered an official, first-party prompting framework. This led us to develop a custom framework, with the goal that it should be possible for auditors to create new prompts without ever modifying Toucan’s code. requires = [“emit-ownership-doc”, “emit-target-contract”,]
name = “Contract Ownership”
scope = “contract”
instantiation_condition = “any(‘admin’ in s.name.lower() or ‘owner’ in s.name.lower() for s in contract.state_variables)” [[questions]]
name = “can-change”
query = “Is it possible to change the `{{ contract | owner_variable }}` variable by calling a function in the `{{ contract.name }}` contract without aborting the transaction? Think through it step by step, and answer as ‘Yes’, ‘No’, or ‘Unknown’. If ‘Yes’, please specify the function.”
is_decision = true [[questions]]
name = “who-can-call”
runtime_condition = “questions[‘can-change’].is_affirmative()”
query = “””To reason about ownership:
1) First, carefully consider the code of the function
2) Second, reason step by step about the question.
Who can call the function successfully, that is, without aborting or revering the transaction?”””
answer_start = “””1) First, carefully consider the code of the function:””” [[questions]]
name = “can-non-owner-call”
runtime_condition = “questions[‘can-change’].is_affirmative()”
query = “Can any sender who is not the current owner call the function without reverting or aborting?”
is_decision = true
finding_condition = “question.is_affirmative()” Figure 2: Sample question chain asking about contract ownership. Before questions are emitted, the prompting framework also emits a specific explanation of what ownership means, with examples and information about the target contract. Our framework supported chaining multiple questions together to support Chain of Thought and similar prompting techniques (Figure 2). Since GPT models like Codex are multi-shot learners, our framework also supported adding background information and examples before forming a prompt. The framework also supported filtering on a per-question basis, as there may also be some questions relevant only to specific kinds of contracts (say, only ERC-20 tokens), and others questions may have a specific scope (e.g., a contract, function, or file scope). Finally, each question could be optionally routed to a different model. The prompting framework also took great lengths to abide by OpenAI’s API limitations, including batching questions into one API invocation and keeping track of both the token count and API invocation rate limits. We hit these limits often and were very thankful the Codex model was free while in beta. Test data One of our development goals was that we would never compromise customer data by sending it to an OpenAI API endpoint. We had a strict policy of running Toucan only against open-source projects on GitHub (which would already have been indexed by Codex) with published reports, like those on our Publications page). We were also able to use the rather extensive test set that comes with Slither, and our “building secure contracts” reference materials as additional test data. It is important to note that some of these tests and reference materials may have been a part of the Codex training set, which explains why we saw very good results on smaller test cases. The missing tools The lack of tooling from both OpenAI and Microsoft has been extremely disappointing, although that looks to be changing: Microsoft has a prompting library, and OpenAI recently released OpenAI Evals. The kinds of tools we’d have loved to see include a prompt debugger; a tree-graph visualization of tokens in prompts and responses with logprobs of each token; tools for testing prompts against massive data sets to evaluate quality; ways to ask the same question and combine results from counterexamples; and some plugins to common unit testing frameworks. Surely someone is thinking of the developers and making these tools? Current programming languages lack the facilities for interfacing with neural architecture computers like LLMs or similar models. A core issue is the lack of capability to work with nondeterminism and uncertainty. When using LLMs, every answer has some built-in uncertainty: the outputs are inherently probabilistic, not discrete quantities. This uncertainty should be handled at the type system level so that one does not have to explicitly deal with probabilities until it is necessary. A pioneering project from Microsoft Research called Infer.NET does this for .NET-based languages, but there seem to be few concrete examples and no real tooling to combine this with LLMs. Prompt engineering, and surrounding tooling, are still in their infancy. The biggest problem is that you never know when you are done: even now, it is always possible that we were just one or two prompts away from making Toucan a success. But at some point, you have to give up in the face of costs and schedules. With this in mind, the $300K salary for a fantastic prompt engineer does not seem absurd: if the only difference between a successful LLM deployment and a failure is a few prompts, the job quickly pays for itself. Fundamentally, though, this reflects a lack of tooling to assess prompt quality and evaluate responses. There is no particularly good way to determine if one prompt is better than another or if you’re on the right track. Similarly, when a prompt fails against an input, it is frustratingly difficult to figure out why and to determine, programmatically, which prompts are merely returning the wrong result versus completely hallucinating and misbehaving. Unit tests are also problematic; the results are not guaranteed to be the same across runs, and newer models may not provide the same results as prior ones. There is certainly a solution here, but again, the tooling developers expect just wasn’t present. OpenAI Evals is likely going to improve this situation. Overall, the tooling ecosystem is lacking, and surprisingly, the biggest names in the field have not released anything substantial to improve the adoption and integration of LLMs into real software projects that people use. However, we are excited that the open source community is stepping up with really cool projects like LangChain and LlamaIndex. Humans still reign supreme OpenAI’s Codex is not yet ready to take over the job of software security auditors. It lacks the ability to reason about the proper concepts and produces too many false positives for practical usage in audit tasks. However, there is clearly a nascent capability to perform interesting analysis tasks, and underlying models should quickly get more capable. We are very excited to keep using the technology as it improves. For example, the new larger context window with GPT-4 may allow us to provide enough context and direction to handle complex tasks. Even though Codex (and GPT-4) do not currently match mature algorithmic-based tools, LLM-based tools—even those of lower quality—may have interesting uses. For languages for which no analysis tooling exists, developers can bootstrap something from LLMs relatively quickly. The ability to provide some reasonable analysis where none previously existed may be considerably better than nothing at all. We hope the ability to integrate language models into existing programs improves quickly, as there is currently a severe lack of languages, libraries, type systems, and other tooling for the integration of LLMs into traditional software. Disappointingly, the main organizations releasing LLMs have not released much tooling to enable their use. Thankfully, open-source projects are filling the gap. There is still enormous work to be done, and whoever can make a wonderful developer experience working with LLMs stands to capture developer mindshare. LLM capability is rapidly improving, and if it continues, the next generation of LLMs may serve as capable assistants to security auditors. Before developing Toucan, we used Codex to take an internal blockchain assessment occasionally used in hiring. It didn’t pass—but if it were a candidate, we’d ask it to take some time to develop its skills and return in a few months. It did return—we had GPT-4 take the same assessment—and it still didn’t pass, although it did better. Perhaps the large context window version with proper prompting could pass our assessment. We’re very eager to find out!
- The models are not able to reason well about certain higher-level concepts, such as ownership of contracts, re-entrancy, and fee distribution.
- The software ecosystem around integrating large language models with traditional software is too crude and everything is cumbersome; there are virtually no developer-oriented tools, libraries, and type systems that work with uncertainty.
- There is a lack of development and debugging tools for prompt creation. To develop the libraries, language features, and tooling that will integrate core LLM technologies with traditional software, far more resources will be required.
Figure 1: True positives (green) and false positives (red) found by Slither, Toucan, and both on some simple re-entrancy tests. The Toucan results are not encouraging. We tried looking at the true/false positive sets of Slither and Toucan, and found that each tool had a different set of false positives/false negatives, with some overlap (Figure 1). Codex was not able to effectively reduce the false positive rate from a prototype Slither integer overflow detector. Overall, we noticed a tendency to reply affirmatively to our questions, increasing the number of positives discovered by Toucan. Codex can perform basic static analysis tasks, but the rate of failure is too high to be useful and too difficult to characterize. This capability to perform successful analysis, even on short program fragments, is very impressive and should not be discounted! For languages that Codex understands but for which no suitable tooling exists, this capability could be extremely valuable—after all, some analysis could be much better than nothing. But the benchmark for Solidity is not nothing; we already have existing static analysis tooling that works very well. How we framed our framework During Toucan’s development, we created a custom prompting framework, a web-based front end, and rudimentary debugging and testing tools to evaluate prompts and to aid in unit and integration tests. The most important of these was the prompting framework. Prompting framework If we were making Toucan today, we’d probably just use LangChain. But at the time, LangChain did not have the features we needed. Frustratingly, neither OpenAI nor Microsoft offered an official, first-party prompting framework. This led us to develop a custom framework, with the goal that it should be possible for auditors to create new prompts without ever modifying Toucan’s code. requires = [“emit-ownership-doc”, “emit-target-contract”,]
name = “Contract Ownership”
scope = “contract”
instantiation_condition = “any(‘admin’ in s.name.lower() or ‘owner’ in s.name.lower() for s in contract.state_variables)” [[questions]]
name = “can-change”
query = “Is it possible to change the `{{ contract | owner_variable }}` variable by calling a function in the `{{ contract.name }}` contract without aborting the transaction? Think through it step by step, and answer as ‘Yes’, ‘No’, or ‘Unknown’. If ‘Yes’, please specify the function.”
is_decision = true [[questions]]
name = “who-can-call”
runtime_condition = “questions[‘can-change’].is_affirmative()”
query = “””To reason about ownership:
1) First, carefully consider the code of the function
2) Second, reason step by step about the question.
Who can call the function successfully, that is, without aborting or revering the transaction?”””
answer_start = “””1) First, carefully consider the code of the function:””” [[questions]]
name = “can-non-owner-call”
runtime_condition = “questions[‘can-change’].is_affirmative()”
query = “Can any sender who is not the current owner call the function without reverting or aborting?”
is_decision = true
finding_condition = “question.is_affirmative()” Figure 2: Sample question chain asking about contract ownership. Before questions are emitted, the prompting framework also emits a specific explanation of what ownership means, with examples and information about the target contract. Our framework supported chaining multiple questions together to support Chain of Thought and similar prompting techniques (Figure 2). Since GPT models like Codex are multi-shot learners, our framework also supported adding background information and examples before forming a prompt. The framework also supported filtering on a per-question basis, as there may also be some questions relevant only to specific kinds of contracts (say, only ERC-20 tokens), and others questions may have a specific scope (e.g., a contract, function, or file scope). Finally, each question could be optionally routed to a different model. The prompting framework also took great lengths to abide by OpenAI’s API limitations, including batching questions into one API invocation and keeping track of both the token count and API invocation rate limits. We hit these limits often and were very thankful the Codex model was free while in beta. Test data One of our development goals was that we would never compromise customer data by sending it to an OpenAI API endpoint. We had a strict policy of running Toucan only against open-source projects on GitHub (which would already have been indexed by Codex) with published reports, like those on our Publications page). We were also able to use the rather extensive test set that comes with Slither, and our “building secure contracts” reference materials as additional test data. It is important to note that some of these tests and reference materials may have been a part of the Codex training set, which explains why we saw very good results on smaller test cases. The missing tools The lack of tooling from both OpenAI and Microsoft has been extremely disappointing, although that looks to be changing: Microsoft has a prompting library, and OpenAI recently released OpenAI Evals. The kinds of tools we’d have loved to see include a prompt debugger; a tree-graph visualization of tokens in prompts and responses with logprobs of each token; tools for testing prompts against massive data sets to evaluate quality; ways to ask the same question and combine results from counterexamples; and some plugins to common unit testing frameworks. Surely someone is thinking of the developers and making these tools? Current programming languages lack the facilities for interfacing with neural architecture computers like LLMs or similar models. A core issue is the lack of capability to work with nondeterminism and uncertainty. When using LLMs, every answer has some built-in uncertainty: the outputs are inherently probabilistic, not discrete quantities. This uncertainty should be handled at the type system level so that one does not have to explicitly deal with probabilities until it is necessary. A pioneering project from Microsoft Research called Infer.NET does this for .NET-based languages, but there seem to be few concrete examples and no real tooling to combine this with LLMs. Prompt engineering, and surrounding tooling, are still in their infancy. The biggest problem is that you never know when you are done: even now, it is always possible that we were just one or two prompts away from making Toucan a success. But at some point, you have to give up in the face of costs and schedules. With this in mind, the $300K salary for a fantastic prompt engineer does not seem absurd: if the only difference between a successful LLM deployment and a failure is a few prompts, the job quickly pays for itself. Fundamentally, though, this reflects a lack of tooling to assess prompt quality and evaluate responses. There is no particularly good way to determine if one prompt is better than another or if you’re on the right track. Similarly, when a prompt fails against an input, it is frustratingly difficult to figure out why and to determine, programmatically, which prompts are merely returning the wrong result versus completely hallucinating and misbehaving. Unit tests are also problematic; the results are not guaranteed to be the same across runs, and newer models may not provide the same results as prior ones. There is certainly a solution here, but again, the tooling developers expect just wasn’t present. OpenAI Evals is likely going to improve this situation. Overall, the tooling ecosystem is lacking, and surprisingly, the biggest names in the field have not released anything substantial to improve the adoption and integration of LLMs into real software projects that people use. However, we are excited that the open source community is stepping up with really cool projects like LangChain and LlamaIndex. Humans still reign supreme OpenAI’s Codex is not yet ready to take over the job of software security auditors. It lacks the ability to reason about the proper concepts and produces too many false positives for practical usage in audit tasks. However, there is clearly a nascent capability to perform interesting analysis tasks, and underlying models should quickly get more capable. We are very excited to keep using the technology as it improves. For example, the new larger context window with GPT-4 may allow us to provide enough context and direction to handle complex tasks. Even though Codex (and GPT-4) do not currently match mature algorithmic-based tools, LLM-based tools—even those of lower quality—may have interesting uses. For languages for which no analysis tooling exists, developers can bootstrap something from LLMs relatively quickly. The ability to provide some reasonable analysis where none previously existed may be considerably better than nothing at all. We hope the ability to integrate language models into existing programs improves quickly, as there is currently a severe lack of languages, libraries, type systems, and other tooling for the integration of LLMs into traditional software. Disappointingly, the main organizations releasing LLMs have not released much tooling to enable their use. Thankfully, open-source projects are filling the gap. There is still enormous work to be done, and whoever can make a wonderful developer experience working with LLMs stands to capture developer mindshare. LLM capability is rapidly improving, and if it continues, the next generation of LLMs may serve as capable assistants to security auditors. Before developing Toucan, we used Codex to take an internal blockchain assessment occasionally used in hiring. It didn’t pass—but if it were a candidate, we’d ask it to take some time to develop its skills and return in a few months. It did return—we had GPT-4 take the same assessment—and it still didn’t pass, although it did better. Perhaps the large context window version with proper prompting could pass our assessment. We’re very eager to find out!
Categorías: Security Posts
