Summary of big data security analysis

Big data is nowadays the hottest IT industry vocabulary, followed by data warehouse, data security, data analysis, data mining, and so on around the large number of commercial value of the use of industry professionals gradually sought after the focus of profit.

I am in communication with the user data problems often encounter some problems, now these common problems summary, initiate, hope to help everyone.

1. What is the core goal of Big data security analysis?

Answer: In order to be able to find hidden in the data behind the security truth. There is a correlation between the data, the traditional analysis can not aggregate the massive data, but the Big data technology can cope with the analysis of massive data needs. Through the Big data base to dig out the APT attack, the network secret channel, abnormal user behavior and other security events. On this basis can be built as a security decision support system for security decision-making to provide data support.

2. Where can I learn about big data security analysis in detail?

   Ans: Big data security is a concerning issue as companies spend a lot of time monitoring and analyzing data about events on networks, servers, and other devices as part of their information security efforts. So, if you are interested in a career in this field, it is recommended to take an online course that covers big data security analytics in detail. In fact, you can pursue a data engineering certification that will demonstrate your in-depth knowledge of this concept.

3. Big data security analysis Enterprise deployment plan and success stories.

Answer: In the national grid, operators have been successful cases. Through the analysis platform to the overall security situation perception, the overall perspective of the overall security situation for macro control. Take the operator as an example, summarize the data of the whole network and carry out data mining work, visualize the results will be presented.

3. Introduction to the development of Big data security analysis at home and abroad.

Response: At present, foreign relatively mature Big data security analysis mainly to Cisco’s Open SOC as the representative. It uses Big data technology to collect network traffic, security device log, business system log, network device log, and these data mining, association and other operations, and finally find out the security incident.

4. Is there a mature methodology for Big data security analysis?

Answer: We want to look at this problem from two perspectives.
First, Big data is a concrete technology implementation. This technology can solve the difficult requirements of traditional data mining in its applicable scenario.
The method of safety analysis has been constantly innovating. There are still some ideas in the safety analysis methodology that can not be landed and the core problem that can not be landed is the lack of technical support.
At present, we use Big data technology is not the safety analysis of innovation, but the safety analysis has been unable to achieve the goal to land. Just as the concept of relational data, which was first proposed in 1970, and landing products in 1976 only the corresponding prototype. Big data technology is the realization of the security analysis methodology.

5. Does the Big data security analysis support the existence of technical standards or specifications?

Answer: There are no technical standards or specifications, but the country is developing the appropriate standards. Green League Technology 2016 will participate in the development of standards.

6. Big data security analysis of the project process is easy to encounter technical difficulties or need a lot of investment links?

Response: Analysis platform is basically a mature technology, the main difficulties in the early planning and security analysis of the two links. Pre-planning to be able to accurately estimate the hardware configuration, storage capacity and other basic information, the latter part of the security analysis of the need for professionals to dig deep data.

7. How to achieve data-driven business security from the perspective of Big data security analysis?

Acknowledgment: A Big data analysis can quantify the security events that exist in the current enterprise and drive the business through security events to achieve the goal of data-driven business security.

8. What are the necessary conditions for achieving a Big data security analysis as a non-IT type enterprise?

Answer: There is a full-time IT team with a full-time security team that has the necessary resources to invest and have the necessary process support.

9. What is the status quo of Big data security analysis visualization? What are the contents, methods, and forms of the show?

Response: Visualization technology has been evolving, and visualization is widely used in BI systems without Big data. With the Big data technology mature, visualization technology can not only achieve the traditional pie chart, line chart, scatter diagram, bar chart, bar chart, but also to map, thermal map, bubble chart, trying, parallel coordinate map And other multi-dimensional display.

10. How can the benefits of Big data security analysis be demonstrated from the presentation level?

Answer: Show just the final result of the security analysis.
The core of the advantage of Big data security analysis is the security analysis model. The benefits of the presentation level are derived entirely from the definition of the security model, and the advantages are not only illustrated from the presentation level. This is mainly because, in the absence of Big data technology before the visual display technology is also rapid development.

11. What are the most commonly used data types for current data security analysis?

Response: DDoS situational awareness, traceability model, APT attack model, asset vulnerability situational awareness, site vulnerability situational awareness, and so on.

12. If the expert system, statistical analysis, machine learning three dimensions to achieve Big data security analysis, whether there is a corresponding algorithm or data model?

Answer: These three are different levels.
The expert system is usually made up of two components online and offline. Offline part of the customer’s local knowledge base, which record a lot of experience, through historical experience to deal with the problem. Online part of the cloud knowledge base system, the customer through the cloud system to ask questions, solve the problem, and the online system is usually 7 * 24 hours, by the global experts to deal with the problem.
Statistical analysis, through simple statistics for data filtering and results presented. Simple data statistics are usually done by non-professionals. Can find some problems from a macro point of view, but can not achieve in-depth data mining work. In order to deal with such a reality, in the business system will build a data warehouse, through the data warehouse to achieve data mining work. However, due to the establishment of data warehouse time and effort, only in a large group of enterprises will be used in the security field.
Machine learning, in fact, self-correction procedures to achieve the accuracy of the results. This is a more mature technology, there are many mature cases in the financial field. Machine learning is mainly used in areas where it is difficult to specify rules, such as abnormal traffic monitoring, abnormal behavior detection. It is usually used in a business scenario where it is difficult to judge by rules.
In these three levels have mature algorithms and applications, and have passed the actual scene of the test.

13. With regard to APT attacks, does the 0day attack have a mature, Big data-based solution?

Answer: APT attacks are generally attacked by attack chain.
The attack chain is divided into three phases:
1. Threats into stage
2. Threat spread phase
3. Data stealing stage.

APT attack detection and defense, focusing on the first two stages, threatening to enter and spread, Big data analysis using threat information system, the network, mail, security, operating system level data summary, statistics, correlation analysis Detect the threat of entering the enterprise. This is the application of Big data analysis in the field of APT threat detection.
For the 0Day vulnerability, Green League more use of the deployment of the network boundary threat analysis system for real-time monitoring, through the static analysis of samples and dynamic analysis to determine whether there is a threat. After the analysis of the sample analysis engine, you can get the sample whether the use of 0Day loopholes, according to the sample’s own reputation information, such as file signatures, samples used to connect to the CnC address, you can use Big data engine, the current data and archived Historical data for analysis, positioning and backtracking affected host, user and other information.

14. What are the Big data-based security analysis algorithms or models that have been implemented for known threat patterns?Response:
    1) attack chain association analysis of the
    same asset, according to the time of threat detection analysis, description of the attack chain2) merge statistics of the
    same type of attack events to merge, many-to-one statistics, one-to-many statistics

    3) threat intelligence association analysis According
    to the threat of intelligence, the current data and historical data recursive query, generate alarm events

    4) abnormal traffic
    learning normal access to traffic, when the traffic is abnormal when the alar