Data Mining A Tool For Tech Development

The use of terms in the literature related to processing patent-related data is quite confusing. The terms web mining, data mining, patent mining, text mining and visualization are employed for the processing of the documents. This chapter will try to give some explanations of the terms and explain why web mining was chosen for the title of the study.

To Place An Order For The Complete Project Material Pay N,5000 To

Guaranty Trust Bank (GTBank)
Acct. Name - Uwadia Eyemeka
Acct. No. - 0127561472

Then Send a text of your names, the topic you paid for, a valid email address to 07036785443

Hand, Mannila and Smyth (2000) define the term data mining as follows: Web mining is the analysis of (often large) observational data sets to find unsuspected relationships and to summarize the data in novel ways that are both understandable and useful to the data owner. Feldman and Sanger (2001) definition of text mining: Text mining can be broadly defined as a knowledge-intensive process in which a user interacts with a document collection over time by using a suite of analysis tools. In a manner analogous to data mining, text mining seeks to extract useful information from data sources through the identification and exploration of interesting patterns. In the case of text mining, however, the data sources are document collections, and interesting patterns are found not among formalized database records but in the unstructured textual data in the documents in these collections.

Clustering is a process which groups the objects into groups called clusters. This is done by classifying the objects. The difference between clustering and categorization is according to Feldman and Sanger (2007) that in categorization problems we are provided with a collection of preclassified training examples and the task of the system is to learn the description of classes in order to be able to classify a new unlabeled object. In the case of clustering, the problem is to group the given unlabeled collection into meaningful clusters without any prior information. Any labels associated with objects are obtained solely from the data.

The term patent mining is quite widely used in the literature referring to processing patent data with data and text mining techniques (Jin et al, (2007)  and (Kasravi, et al 2007). It refers to data mining of patent documents. The idea of visual data exploration, visualization, is, according to Keim (2002), to present the data in some visual form, allowing the human to get insight into the data, draw conclusions, and directly interact with the data. The development of visualization techniques in the last decade has made it possible to widen the visualization of low dimensional data, e.g. making histograms of yearly attributes, to create sophisticated visualizations of high-dimensional text data.

Patent documents contain structured and unstructured data alike; they are semi structured as Feldman and Sanger (2001) write. The bibliographic information of a patent is structured and follows a strict format. For example, it contains patent assignee and inventor names, different identifiers such as priority and publication numbers, years and classifications. Unstructured data is text explaining the invention and the extent of protection of the patent, e.g. title, abstract and claims. According to Tseng et al (2007) visualizations of results of patent analysis are called patent graphs if they are prepared from structured data, and patent maps if they are from unstructured texts. However, they admit that, loosely speaking, the term map is used often for both cases.

How does mining really work? Let’s look on how Data Mining (DM) works. It’s regarded as the analysis step of the Knowledge Discovery in Databases (KDD) process (Chidanand et al, 2002). KDD usually has three steps, preprocessing, then DM, then finally data verification (Usama et al 1996). For DM, it uses data stored in data warehouses for analysis. DM technologies use Artificial Intelligence AI and neural networks; a good review of neural networks applications in business is found in (Vellidoa et al, 1999).  AI and neural networks are nonlinear predictive models. Doug (2011), learns through experience and training. Furthermore, AI techniques are highly used in business models and predictions (Meryem et al, 2010). AI led to the more advanced technique of machine learning. Machine learning is the ability of the machine to adapt and learns from previous trials and errors, and then it tries to find out how to be more effective. Other techniques are decision trees and genetic algorithms. Quinlan, (1986) decision trees are top down induction rationale thinking tools, they support classifying the decisions into different branches. Starting from their roots and stemming to the leaves, each decision branching out has risks, possibilities, and outcomes. Certain decision trees type used are Classification and Regression Trees CART. DM tasks are several and depend on the different fields where the DM is applied. Classifying data stored in multi-dimensional databases is a prominent task. Classifying involves identifying all groups that can be found in the data, like grouping fraudulent transactions in a separate group from legitimate transactions. Associations and rule inductions, intelligently inferring if-then relations from patterns found hidden in the data. This leads to finding hidden correlations, like market baskets, which are the products bought mostly together. Another task is regression modeling or predictive modeling (Dave, 2010), which helps in predicting future trends. Usually regression is used for extrapolating data in mathematics, in DM; it helps to find a model that fits dataset.

Data visualization, visually aids in linking multi-dimensional data together, such as Exploratory Data Analysis EDA and model visualization. Data visualization tasks are emerging task of newer interactive DM tools. Other tasks include Anomaly detection, were anomalies are caught. Summarizations that express information extracted in a compact form; and aggregation of data, where sums of data are compacted to single figures or graphs (Michael and Gruenwald, 1999). Data Mining requires huge computational resources. DM requires as a perquisite a data source, usually a data warehouse, or a database. Inmon (1996) Data warehouses are large databases used for data analysis and aggregation. It extracts and transforms the data from the DBMS the enterprise use for daily activates and through Extraction, Transformation, and Loading ETL processes (Rajender, 2008). ETL extracts the data from the DB, then it pre-process the data and finally load it into the data warehouse for further processing of this cleaned up data. Aside from this, DM requires substantial processing power, usually top class server level computing prowess. A growing interest in combing data mining with Cloud Computing technologies is emerging such as providing DM as a service, such as found in (Marinela et al, 2007). DM applies number crunching algorithms, parallel processing, and neural networks with AI techniques, thus requiring this huge computational resources. For standardizing DM, the CRoss Industry Standard Process for Data Mining or CRISP-DM for short, was developed (CISPDMB, 2008). CRISP-DM is standard model developed in 1997 by the ESPRIT funding initiative, as part of a Euro Union project, and lead by leading industry companies such as SPSS, now part of IBM, and NCR Corporation. Until data, CRISP-DM is the leading standard for DM, as most leading DM software implements it.

ALSO READ  Detection and Control of Cyber Crime

The next question is how does TM work? TM essentially is based on how do we read and comprehend text. This process of reading, then understanding what is read is to some extent imitated. However, this is not a very easy process for computers. For TM techniques, it first has to retrieve relevant documents, data, or text found on the WWW. For this step Information Retrieval IR systems are used (Judy and JISC, 2006), Google search engine is an example of such systems. The second process is NLP; this is the most difficult part of TM. In NLP, AI and neural networks are again used to parse the text the same way humans do to comprehend the text. The text is parsed; nouns and verbs are used to understand grammatically the meaning on each sentence. The final step is information extraction; here linguistic tools are used to get information from the comprehended text. Entities, characters, verbs, and places are correlated and hidden unknown new information is generated. This is where DM techniques are applied at the final stages to extract the information. This is how most TM tools work.

TM techniques try to emulate human comprehension of textual data. TM highly used in many applications to replace manual search in textual documents by humans. TM techniques, unlike DM techniques, deal much with unstructured data sets, thus more complicated. Healthcare and medical usage includes linking several hundred of medical records together, finding relations between symptoms and prescriptions (Dave, 2010). Media applications also use TM, especially in the political aspects of certain controversial issues or controversial characters. TM is also used for clustering archived documents into several clusters according to predefined semantic categories (Neto et al, 2000). TM is newly finding new uses in the legal and jurisdictional fields, as TM is applied to patents, and criminal profiling. TM is used in text summarizations, where it effectively identifies the main names, characters, verbs, and the most used words or referenced subjects in large documents. TM is also used in TM OLAP, found in, as a textual search tool rather than numerical. Furthermore, Byung-Kwon and Il-Yeol (2011), TM is an essential part of modern IR engines, such as Google’s search engine and Yahoo’s search engine, as they apply sophisticated TM techniques to correlate search queries together.

The final question is how does WM work? WM is based on Internet and agent technologies (Kosla and Blockeel, 2000), that utilize soft computing and fuzzy logic techniques. Again, WM technologies rely on IR tools to find the data it needs. IR systems provide means and ways in which these intelligent agents can scour the WWW. In addition, it is worth noting that such IR systems are greatly supported by the developments of Semantic Web technologies, (Rafael et al, 2012) SW technologies were developed to build semantics over web published content and information on the WWW, in the aim of easing the retrieval process of the content by humans and machines. As SW technologies are still a work in progress, as more new web technologies are implementing SW techniques to aid in the search process of such contents. The next step is the agents programming, these agents are programmed after finding their targets to apply their mining techniques. These include analyzing the HTML document, parsing, and extracting all the hyperlinks, multimedia among other things. Users’ preferences and online accounts are specifically tracked to identify their sessions and transactions logged in that system. Server data, like site traffic, activates and even possible proxies are retrieved for further analysis. Final step for these agents is to analyse the gathered data, using DM techniques, to understand the habits, patterns found in the WWW.

WM tasks depend on the mining purpose intended. WM is divided into three main categories as in (Sankar et al, 2002), web content mining WCM, web structure mining WSM, and web usage mining WUM. WCM is intended for retrieval can fetch and locate context sensitive text, multimedia, and hyperlinks depending on the giving context. Other WM tasks intended for WSM include finding the chain of links or site maps of certain sites, mainly to find were most of the traffic is headed. Finally, WM intended for WUM can collect logs, cookies, bookmarks, and even browsers history and metadata of targeted users. WUM is also used for mining social networks, namely online blogs (Federico and Pier, 2005). These tasks are all WWW oriented, but WM also plays a key role in mining social media networks for investigations, intranets, and to lesser extent Virtual Private Networks VPNs. A new use of WM is multimedia mining, where WM tools will try to mine out multimedia as pictures, movies, audio files, and applications.

Business applications rely on mining software solutions. Mining tools are now an integral part of enterprise decision-making and risk management. Acquiring information through mining is referred to as Business Intelligence BI. Enterprise datasets are growing rapidly, thanks to use of Information Systems IS, and data warehousing. On average, credit card companies usually have millions of transactions logged per year. Largest data sets are usually generated by large
telecommunications and mobile operators as they mount up to 100 million user accounts generating thousands of millions of data per year Clifton Phua, Vincent, Kate and  Ross, (2010).

As these number mount up, analytical processing such as OLAP and manual comprehension seems ineffective. With BI such tasks are within reach. According to Gartner Group “Data mining and artificial intelligence are at the top five key technology areas that will clearly have a major impact across a wide range of industries within the next three to five years,” this was back in a 1997 report Sang and Keng (2001). Also according to Gartner Group reports in 2008, it was found that 80% of data found in enterprises information systems is unstructured, and would very likely to double in size nearly every three months. BI has become the prevalent decision support systems in organizations. BI has dominated many industries including retail, banking, and insurance. The First American Corporation FAC is an example of a success story in implementing BI to improve its customers’ loyalty and better investment. It’s worth mentioning that BI software is aimed at knowledge workers, mainly executives, analysts, middle management, and to a lesser extent operational management.

BI is implemented through mining tools; these tools generate findings that are ultimately used to gain competitive advantage over rivals, better and efficient business operations, and better survivability and risk management. Mining tools provide better customers’ relationship management CMR, through mining real habits, patterns, and even customers churn. Customers churn is defined as the per cent of customers that have left the enterprise, most likely to other rivals or due to the inability to keep your competitive advantage and customer satisfaction levels.
Habits and trends in customers’ data help in discovering the customers’ segmentations, what customers to target, especially alpha customers.

ALSO READ  Electronic Banking and Cybercrime in Nigeria

Alpha customers are those that play a key role in a product success, thus finding what they want is essential. This means that, mining tools are International Journal of essential for catalogue marketing industries and advertising agencies. In addition, mining tools, especially DM, provides market basket analysis that helps the discovery of products that are bought usually together. As modern economies around the world today are driven by information, becoming information and knowledge based economies; BI tools are from the top reasons of development information technologies in business today. BI tools in business today are integrated in most enterprises tools such as Enterprise Resource Planning ERP tools, Customer Relationship Management CRM tools, supply chain management tools, data warehouses, and even RDBMS. BI is also the main tools for decision support in modern enterprises. BI tools provide competitive advantages, better customer relationships managements, and better management of risk in investments.

Their solution consists of smart BI agents that buy and sell stock autonomously, with human monitoring for erroneous errors. Google, the technology giant, uses BI on its Google Finance service. The Google Finance web page contains dynamic charts of international stock markets, with references to critical points in the graph directing to web pages that are the service got its information, providing assurance to end users. Mining tools are also used for automatic spam detection, and the defence against fraud, through fraud detection techniques utilizing mining tools. Most major banking and telecommunication companies apply automated fraud detection systems through mining techniques, AT&T, bank of America are examples of such users of fraud detection. In the next subsections we will look at the main aspects were BI through mining tools is used to gain business proficiency. BI and mining tools are used exchangeable in the following text.

Most work considers each mining technique separate from one another. In Michael and Le (1999) the researcher has provided an overview of Knowledge Discovery in Databases (KDD) approaches. They also classified the approaches depending on software characteristics. In Chidanand  et al (2002) the researcher demonstrated how modern technologies shifted the process of decision-making, from manual data analysis using modeling and stochastic to an automated computer driven process. The researcher also stated that knowledge discovery tools have benefits such as increased profitability. In addition, risk management and market segmentation is another advantage. A survey of visual data mining techniques is found in (Maria et al, 2003). The researcher has stated that large data sets with complex dimensions need a better way for representation. In their paper, the researcher has reviewed previous work done in data visualization (Chi, 2000). The researcher classified data visualization techniques into six different classes, based on the parameters of the data.

In Hotho, Nu¨rnberger and Paar, (2005) the researcher has surveyed the relatively young and interdisciplinary field of TM. Since most information found in computerized form are textual, the need to extracts this unstructured text into informative knowledge demands new tools. TM tools are machine tools that analyse written text with a certain context (Feldman and Dagan, 1995). A case study of TM is found in Michele, Giorgio and Roberta (2003), the paper discusses the use of TM for patent analysis. The authored discussed how professional patent information business is sceptical in using TM tools. The paper discussed showcased PackMOLE (Mining Online Expert on Packaging Patents), a TM tool, designed for mining patent information in the packaging field. The researcher showed that PackMOLE tool has advantages over the manual patent portfolio analysis techniques. However, the tool calibration of its internal clustering processes is difficult, and consumes time. This leads to the use of a hybrid use of text mining techniques and manual patent classifications in conjunction.

In Vishal and Gurpreet (2009), the researcher presented a review of TM techniques. The researcher clearly stated that TM faces challenges as natural language processing NLP techniques are not readily made for mining activities. The paper illustrated several TM technique that included information extraction, topic tracking, summarization, categorization, clustering, concept linkage, information visualization and question answering to name a few. Finally, the author stated that TM is used in media, banking, politics, and even in insurance. How business intelligence is derived from web mining is found in (Ajith, 2003). WM or web usage mining is described as an intelligence tool to aid enterprises in the intense competition found in ecommerce.


A theoretical framework is a collection of interrelated concepts, like a theory but not necessarily so well worked-out. A theoretical framework guides your research, determining what things you will measure, and what statistical relationships you will look for. Theoretical frameworks are obviously critical in deductive, theory-testing sorts of studies. In those kinds of studies, the theoretical framework must be very specific and well-thought out. The “theoretical framework” of an experiment or paper refers to the larger assumptions in which the researcher is working. A theoretical framework provides a large, overarching structure of ideas that the researcher can then draw from in beginning to analyze a phenomenon or a text.

The enabling technology of Web mining may sometimes called siftware. The technology in other word refers to as mining tools that varies depending on their sophistication, as state of the art tools are expensive.  The enabling technologies are that software that is used for mining of data. These are as follows:

(i)    IBM SPSS is an example of business intelligence software, offering mining capabilities.
(ii)    Clementine was a graphical and widely used DM tool of the late 1990’s, it is precursor to SPSS. IBM also provides online services for Web Mining, called Surfaid Analytics; they provide sophisticated tools for Web Mining. IBM is one of main providers of solution-oriented packages such as IBM’s Cognos 8 solutions.
(iii)    Oracle Data Mining, a part of the company flagship RDBMS software.
(iv)    Service Oriented Architecture (SAS) offers its SAS Enterprise Miner, a     part of its enterprise solutions.
(v)    Microsoft offers SQL Server Analysis Services, a platform dependant solution integrated in Microsoft SQL for Microsoft Windows Server. Other Microsoft products include PowerPivot, a mining tool for small and middle size enterprises. Other open source tools include RapidMiner and KNIME. Situation with open source mining tools however is not as other open source software, as their use is quite limited. Furthermore, with huge decrease of in costs of storing and acquiring data from WWW, data acquisition tools such as RFID tag readers and imaging devices, e-commerce, and telecommunications, mining software costs of procuring dropped considerably.

Competitive advantage is driven by competitive pressure. According to Thiagarajan, Mary and Anna (2011), competitive pressure is degree of pressure that companies feel from rivals and possible new entrants. This pressure is lessened by gaining a competitive advantage. For gaining competitive advantages, enterprises develop market research groups that analyse the large data sets to acquire knowledge. Market research, through mining, try and find what products dominate the market, why this is and what hidden elements that set such products leading in sales. For example, media networks use mining in their market research to set the common factors between audience and the program’s scheduled slot. BBC, used to hire human experts to schedule its programs slots, now its uses fully automated mining tools for scheduling, the results were equivalent or better than the human manual scheduling. Marketing use-mining tools to get the market’s baskets, as mentioned before, market basket are associations of certain products that are highly likely bought together.

ALSO READ  Computer Forensics: Towards Creating a Certification Framework

No competitive retail enterprise is without its set of market baskets, leading in this segment are Wal-Mart, Costco, and K-Mart.

Figure 1: Data Transforming into Business Intelligence

For better risk management, in banking for example mining tools are used to automate risk of bankruptcy. An example is bankruptcy predictions Amir (2001). Bankruptcy predictions are essential risk assessment processes, nearly all large banks such automated tools in their bankruptcy predictions. Most of these tools use neural networks in assessing the counterpart’s liability to pay back the loan given. Bankruptcy prediction tools help the bank in reducing the risk of lending money to trouble or would be troubled customers. These tools also depending on the risk the bank is willing to take, will estimate the fair interest rates depending on the credibility of the other party. In addition, bankruptcy prediction tools are also used to assess the credit loss of current and ongoing bank’s loan portfolio. Credit loss is the prediction of possibility that depreciation or loss levels of loan will be more than the expected 1% in a predefined time frame.

These tools came to the foreground after strict regulations especially after the 2008 market crash and following economic crisis that led many banks to file for bankruptcy. Banking sectors today utilize BI with bankruptcy tools together to create sophisticated profiles on their potential customers before any lending. Usually banks have setup return/risk parameters depending on the other party’s market worth, market share, and current debt. Amir (2001), bankruptcy tools are also used by accounting companies. Auditing now takes into account the possibility of going bankrupt in the near future. Lawsuits are filed against accounting companies if they did not warn the contracting party of its possible bankruptcy. J.P. Morgan utilize its’ own bankruptcy tool called Credit Metrics, it model the change in the credit quality ratings, it can obtain a possible estimate of the risk incurred in the loan.

In manufacturing industries, sophisticated Management Information Systems MIS are used to automate the processes. Such MIS tools generate reports that can be feed to BI tools to better optimize the manufacturing processes. MIS tools are used to automate the work, but such tools were not meant to monitor the performance nor smartly detect how to use the operation data to further develop and optimize the manufacturing process. MIS software for the manufacturing industry includes Material requirements planning MRP systems, which detects when the finished products will be needed or shipped out of the factory. Other example of MIS software for manufacturing is Just in time JIT inventory systems, precise and immediate delivery of materials before usage. These tools also generate huge amount of raw data, reports, and schedules, which could be used for mining.

MAIA Intelligence (2009),  BI tools using such data can generate adaptive manufacturing schedules to match the demands. BI tools help the top management further to lessen the wastage and unnecessary expenditures. BI tools not only improve the manufacturing processes in the physical factory, but also optimize pricing analysis, by examining sales, production, and quarterly reports, the manufacturing enterprise can adjust its pricings on its items. Other optimization of other processes include warranty analysis, by examining the warranty claims and lawsuits occurring from manufacturing errors, failing parts, and possible risk factors. As such, the manufacturing enterprise can reduce such errors, forecast possible risks and avert flawed warranty strategies.

Business Intelligence (BI) tools through its sifting capabilities can locate certain hidden patterns found in daily transactions. These patterns are used for possible fraud and anomaly detection. Fraud and anomaly detection is defined as detection of deceptive transactions or strange and unusual transactions that need inspection. BI tools are one of the main tools used in forensics to detect fraudulence. Most insurance and telecommunication companies use fraud detection daily. As it’s the norm to investigate suspicious transaction for possible fraudulence or anomalies. As such, some businesses are plagued by fraudulence and possible anomalies in their transactions.

Telecommunications companies and credit industries are some of the most plagued industry, since it is hard to detect such fraudulence in these businesses. On the other hand, anomalies are not deliberate actions like fraudulence. They are unusual behaviours that may manifest in normal data for unknown reasons, corrupted data, glitches, or network errors in transmissions. If not identified correctly, anomalies can lead to a shift and possible divergence in finding real frauds.

Anomalies must be detected and left outside the data set, as to leave out the surge or upheavals these anomalies bring in the data. Fraudulence is however deliberate actions are, usually caused for monetary and financial gains, and is usually carried out by white-collar criminals, insiders, high profile technology criminals, and expert computer hackers.

Usually credit companies try to find certain patterns of charging for its customers. For example, usually stolen credit cards will result in erroneous behaviours and transactions. These transactions usually happen in short periods, within few hours; huge sums of figures in thousands to millions are spent in numerous transactions. The fraud detection used to be manual, through reviewing record manually. As BI fraud detection can easily spot such anomalies. Fraud detection is not only limited to fraudulent customers, but to also fraud within the company, as in fraudulent reports and predictions. BI tools detect suspected behaviours over periods of time, providing what accounts or individuals that require special intention. BI is also used in financial crimes like money laundering, through sifting records of individuals, and insider trading, trading upon secret inside information, Intrusion detection, by outsiders into the system, and spam detection, a major problem facing many enterprises email systems. Finally, we can say that the main problem with such tools would be the false positives that can occur with high percentages in some cases, and miss predictions that have legal and monetary consequences.

To Place An Order For The Complete Project Material Pay N,5000 To

Guaranty Trust Bank (GTBank)
Acct. Name - Uwadia Eyemeka
Acct. No. - 0127561472

Then Send a text of your names, the topic you paid for, a valid email address to 07036785443
To Place An Order For The Complete Project Material Pay N,5000 To

Guaranty Trust Bank (GTBank)
Acct. Name - Uwadia Eyemeka
Acct. No. - 0127561472

Then Send a text of your names, the topic you paid for, a valid email address to 07036785443

Speak Your Mind