QueryPie’s New Standard for Penetration Testing

QueryPie has developed and operates a comprehensive penetration testing framework. This framework is built upon globally recognized standards such as NIST SP 800-115 and the OWASP Testing Framework, ensuring a more systematic and thorough penetration testing process. This framework is designed to maximize effectiveness during the limited internal in-house penetration testing time, enabling the efficient detection and analysis of vulnerabilities.

The framework standardizes the penetration testing process and clearly defines the activities at each stage to ensure consistent security verification. As a result, QueryPie maintains a high level of security with each product release, ensuring that any vulnerabilities discovered are effectively addressed. The QueryPie penetration testing framework consists of the following six stages.

QueryPie Penetration Framework V1

QueryPie Penetration Maturity Model?

As the penetration testing process became stabilized within the organization during the security review phases of development, we created the penetration testing maturity model outlined below. This model serves as a means to measure our current level and establish a benchmark for continuous improvement. As a result, we have assessed QueryPie’s penetration testing maturity level as being at the Proactive stage. However, we are not satisfied with this level and are working toward elevating it to the Optimized stage by integrating with the DevSecOps pipeline. This integration aims to shift our security approach from reactive to proactive, and by combining automated penetration testing with AI-based threat detection systems, we expect to make security reviews and responses more efficient and consistent.

Furthermore, the transition to the Optimized stage is not merely a technological advancement. It represents QueryPie’s commitment to providing the highest level of security to our customers, maximizing the reliability of our products, and establishing a system that can stay one step ahead of emerging security threats.

In-house Red Team

The Red Team conducts internal penetration testing and checks vulnerabilities (*CVE, CWE, CCE) at every stage of the CI/CD pipeline. They also perform security reviews and provide guidance throughout the development lifecycle. Before each new version release, the Red Team follows the process outlined below, in collaboration with the QA team and bug bash sessions.

1. Pre-pentest Review: The PM team discusses the key features of the new version and the primary checkpoints.
2. Penetration Testing: Based on the pre-pentest review, the team prioritizes testing key features and conducts a comprehensive penetration test on all functions. (2 weeks)
3. Security Team Internal Review: Any vulnerabilities discovered are initially reviewed by the internal security team.
4. Sharing with Development Team & Assigning Responsibilities: Vulnerability information is shared with the development team, and specific issues are assigned to the responsible parties via Jira tickets for tracking and management.
5. Vulnerability Remediation Verification: Once vulnerabilities have been addressed, the Red Team member verifies the implementation and marks the vulnerability ticket as resolved.

*CVE : Common Vulnerabilities and Exposures *CWE : Common Weakness Enumeration *CCE : Common Configuration Enumeration

The tools and methodologies used by the QueryPie Red Team for testing are as follows. By employing these methodologies, the team standardizes testing, systematically evaluates the severity and priority of vulnerabilities, and establishes effective responses.

Diagnostic Tools

Diagnostic Tools

• Burp Suite
• Neuclei
• OWASP ZAP
• Nessus
• dnSpy
• Snyk
• Github Advanced Security

Diagnostic Items

• OWASP Top 10
• OWASP API Security Top 10
• SANS Top 25
• NIST SP 800-115

Risk Classification

• The CIA individual scale and weight calculation are used for classification
• Weights (Considering the characteristics of the QueryPie product):
  • Confidentiality (C): 0.4
  • Integrity (I): 0.35
  • Availability (A): 0.25
• Example: For a vulnerability with C = 3, I = 1, A = 1, the score is calculated as follows: Score = 3 x 0.4 + 1 x 0.35 + 1 x 0.25 = 1.8

CIA Scale Table

In 2024, the QueryPie Red Team conducted thorough in-house penetration testing for each new version release, discovering a total of 26 vulnerabilities through 7 rounds of testing.

Through these continuous security checks, we are minimizing potential threats and striving to provide a safer, more trustworthy service for our customers.

QueryPie Bug Bounty Program

QueryPie demonstrates an open and proactive approach to security through its Bug Bounty Program.

By responding quickly to vulnerabilities identified by customers and officially acknowledging them, we strengthen the trust relationship with our customers. Ultimately, this trust-building plays a key role in improving the long-term quality and security of our products.

The QueryPie Bug Bounty Program follows the process outlined below:

1. After a bug bounty report is submitted, we reproduce the vulnerability based on the report.
2. If the vulnerability is accepted, a reward is determined according to the Score-based Reward Table.
3. The vulnerability is shared internally for remediation.
4. Rewards are paid, and the reporter is listed in the Hall of Fame. If the reporter wishes, they can also receive a CVE, which plays a significant role in acknowledging their efforts and increasing their recognition.

Bug Bounty Process

QueryPie Hall of Fame

Collaboration with External Penetration Testing Experts

In addition to the regular penetration testing conducted by the in-house Red Team, QueryPie also actively collaborates with independent external security experts to conduct third-party penetration tests. Independent security professionals bring a fresh perspective to assessing the security of QueryPie and offer the following advantages:

"Objective Security Validation"

External experts, not directly involved in the product development, may have limited product understanding but can approach the task with an unbiased perspective as testers. They can identify potential vulnerabilities and apply various attack scenarios. This objective viewpoint enhances the level of security validation and helps identify security flaws that the QueryPie team might have missed.

"Expertise of External Penetration Testers"

External experts are active penetration testers in the industry, with extensive experience in testing various products. Their deep research and understanding of the latest cyber threats and attack techniques allow them to test the product’s defense in ways that QueryPie might not have anticipated. This enables QueryPie’s security team to thoroughly analyze vulnerabilities found by external experts and strengthen the security architecture along with remediation efforts.

"Collaboration with the In-House Team"

External evaluations play a crucial role in complementing and verifying the efforts of the in-house Red Team. For example, vulnerabilities identified by the in-house team, as well as products before new version releases, are revalidated by external experts, allowing QueryPie to iteratively improve the product’s security. The results from external testing serve as an objective performance metric and provide valuable insights for the in-house Red Team to continuously refine QueryPie’s security strategy.

Independent third-party penetration testing is an essential factor in building trust with QueryPie’s customers. Through this process, customers can be confident that QueryPie’s products undergo rigorous security validation, and the QueryPie team can confidently offer a robust PAM solution capable of protecting their critical systems and data.

Best Practices

"Strengthening Secure Coding and Establishing a Security Culture through Penetration Testing"

Penetration testing has greatly contributed to strengthening secure coding in the development phase. For example, as developers directly experience the importance of penetration testing and security reviews, their awareness of security vulnerabilities has significantly increased. As a result, developers now naturally consider security when designing features or writing code and take prompt action to address any vulnerabilities found. In particular, the culture of proactively conducting security reviews related to penetration testing during development and minimizing potential risks has been established across the organization. This ensures that security vulnerabilities are proactively reviewed every time new features are developed, ultimately improving the stability and reliability of the product. This shift has helped improve the overall security level of the organization, contributing to providing safer products to customers.

• Q: Should a security policy be applied to the Test Connection function?
  • To access the Server Groups page, SAC administrator privileges are required, so it was believed that not having a related security policy wouldn't be a major issue.
  • However, due to the absence of a limit on the number of Test Connection failures, brute force attacks could potentially reveal server account passwords, leading to a request for a security review.
• A: Upon review, having SAC administrator privileges does not eliminate security risks. Currently, even with server admin privileges, the passwords for registered servers cannot be viewed in plaintext. However, the following risks exist, so a limit on the number of failed attempts (e.g., 5) is necessary:
  • A server admin may attempt brute force attacks on servers they do not manage.
  • If an admin account is compromised, there is also the risk of server account compromise.

"Objective Security Evaluation and Trust Enhancement through External Penetration Testing Consulting"

Through external penetration testing consulting, we were able to receive objective and reliable evaluations of our product. By thoroughly reviewing the product and validating its security level, external experts provided our internal team with an additional confirmation of the product's security and helped increase trust. This external verification process also boosted customer confidence in our product, while reinforcing our internal confidence in the security level of the product.

"High-Level Technical Expertise Gained through Bug Bounty"

The bug bounty program has provided an opportunity to experience new inspection methods and various external penetration testing approaches, which have been invaluable in strengthening our in-house penetration testing capabilities. This learning and experience help us identify vulnerabilities that might have been overlooked during internal tests, significantly enhancing the depth and quality of our checks. As a result, exceptional penetration testing skills have been internalized within the organization, and over the long term, our ability to strengthen security and respond to issues has greatly improved.

Through this integrated security review, we identified a total of 44 vulnerabilities this year, and aside from 13 ongoing fixes, all vulnerabilities have been addressed.

2024 Security Assessment Results

Conclusion and Moving Forward

QueryPie deeply understands the importance of security and is continuously innovating and evolving to maintain high security standards. With the goal of advancing to the Optimized stage, we are integrating automated penetration testing and AI-based threat detection systems to enhance both the efficiency and effectiveness of our security measures. These efforts go beyond just responding to current security threats; they allow us to proactively identify and respond to potential threats before they materialize.

Moreover, by building and refining our DevSecOps pipeline, we are naturally integrating security tests into the development process, further strengthening the safety of our applications. The introduction of automated fuzzing tests helps identify runtime issues early, ensuring the stability of our services.

This comprehensive and systematic approach to security forms the foundation for providing more reliable services to our customers, allowing them to use QueryPie with confidence. Moving forward, QueryPie will continue to prioritize customer trust and work tirelessly to maintain the highest level of security.