This invention concept would allow big data companies or even government agencies to have the ability to provision storage systems to artificial intelligence, or machine learning algorithms, that automatically categorize, and store information based on fields that are common or identifiable based on the application domain and the structure of the systems that are being managed. This would essentially allow the complete automation and autonomy of data storage systems, using big file-based databases, that act like a giant meta collection system for easy searching of lots of files on the fly to find specific needles in large haystacks, especially for systems that work to prevent terrorism or crime that have significant amounts of surveillance data. An example of this system would be the National Security Agency’s collection of internet records through PRISM, collection of telephone meta data, and collection of other types of Signals Intelligence (SIGINT).
The systems would need to rely not only on their fast data access capabilities, but also searching abilities, so that government workers can locate the specific data sets that they are looking for in a relatively short amount of time. The artificial intelligence allows the storage system to grow over time, without slowing it down. Systems technology with slower access rates can be phased out over time, while newer faster systems are brought online for mission critical data that is collected in real time, in addition to case data that could span 5 years or more, depending upon the case.
The artificial intelligence would also keep security over all records, such that information at higher classification levels would be encrypted with different key sets, that makes it impossible for lower security classification employees to access that information. This security would be on a microscopic level, which may allow certain parts of certain files to be searchable, while disallowing the viewing of the full file for security purposes, such as information pertaining to the location of human assets, collection methods, and even specific information within a file. The artificial intelligence system would be able to determine what information should be classified at what levels, reducing the amount of time that government employees spend on security classification tasks for specific data sets.
