搜档网
当前位置:搜档网 › 支持向量机的股票决策Automatic stock decision support system based on box theory and SVM algorithm

支持向量机的股票决策Automatic stock decision support system based on box theory and SVM algorithm

支持向量机的股票决策Automatic stock decision support system based on box theory and SVM algorithm
支持向量机的股票决策Automatic stock decision support system based on box theory and SVM algorithm

Automatic stock decision support system based on box theory and SVM algorithm

Qinghua Wen *,Zehong Yang,Yixu Song,Peifa Jia

State Key Laboratory on Intelligent Technology and Systems,Tsinghua National Laboratory for Information Science and Technology,Department of Computer Science and Technology,Tsinghua University,Beijing 100084,China

a r t i c l e i n f o Keywords:

Stock data mining Stock trading system Box theory

Support vector machine

a b s t r a c t

The stock market is considered as a high complex and dynamic system with noisy,non-stationary and chaotic data series.So it is widely acknowledged that stock price series modeling and forecasting is a challenging work.A signi?cant amount of work has been done in this ?eld,and in them,soft computing techniques have showed good performance.Generally most of these works can be divided into two cat-egories.One is to predict the future trend or price;another is to construct decision support system which can give certain buy/sell signals.In this paper,we propose a new intelligent trading system based on oscil-lation box prediction by combining stock box theory and support vector machine algorithm.The box the-ory believes a successful stock buying/selling generally occurs when the price effectively breaks out the original oscillation box into another new box.In the system,two SVM estimators are ?rst utilized to make forecasts of the upper bound and lower bound of the price oscillation box.Then a trading strategy based on the two bound forecasts is constructed to make trading decisions.In the experiment,we test the sys-tem on different stock movement patterns,i.e.bull,bear and ?uctuant market,and investigate the train-ing of the system and the choice of the time span of the price box.The experiments on 442S&P500components show a promising performance is achieved and the system dramatically outperforms buy-and-hold strategy.

ó2009Elsevier Ltd.All rights reserved.

1.Introduction

The study of stock market is a hot topic,because if successful,the result will transfer to fruitful rewards.A signi?cant amount of work has been done in this ?eld.Most of the works are the com-bination of soft computing technology and technical analysis in stock analysis.They can generally be divided into two categories.One is to predict the future trend or price in the next day or next few days based on the historical prices and the technical indicators.Among them,as claimed by Grudnitski and Osburn (1993),arti?-cial neural networks (ANN)are particularly well suited for ?nding accurate solutions in an environment characterized by complex,noisy,irrelevant or partial information.So many works focus on applying different neural networks into stock prediction (Chen &Leung,2004;Kwon &Moon,2007;Lin,Yang,&Song,2008).Re-cently,a new type of learning machine,called support vector ma-chines (SVM),has been receiving increasing attention in areas ranging from its original application in pattern recognition to the extended application of regression estimation.Cao and Francis (2003)show that SVM forecasts signi?cantly better than the BP

network in ?nancial time series forecasting.Bao (2005)concludes the support vector machines is a robust technique for stock index regression.Other methods include,hidden markov model (HMM)(Hassan &Nath,2005),reinforcement learning algorithm (Lee,2001).

Another high-level research is to develop decision support sys-tem which helps to make trading decision.A typical work is (Baba,Inoue,&Yanjun,2002)in which a decision support system was proposed by utilizing the neural network to make a forecast of the TOPIX in the future and the genetic algorithm to ?nd an effec-tive way of dealing.In more recent work (Li &Chen,2006),based on the mechanisms of group decision making and cooperative learning,the proposed system combines of Wilson’s XCS and neu-ral network and the investment strategy was retrieved from the cooperative learning of 50agents.The problem of all of these methods is they did not use or used little the practical knowledge and techniques accumulated in stock investment which we be-lieved are critical to design a workable stock expert system.

In our previous work (Bao &Yang,2008),an expert system which learn trading strategy by markov network probabilistic model from high-level representation of time series turning points and technical indicators was developed,which is based on the idea of price trend turning point in technical analysis.In this paper,the objective of this paper is to exploit the common used box theory of stock and develop an intelligent stock decision

0957-4174/$-see front matter ó2009Elsevier Ltd.All rights reserved.doi:10.1016/j.eswa.2009.05.093

*Corresponding author.

E-mail addresses:wenqh06@https://www.sodocs.net/doc/3e4030327.html, (Q.Wen),yangzehong@https://www.sodocs.net/doc/3e4030327.html, (Z.Yang),songyixu@https://www.sodocs.net/doc/3e4030327.html, (Y.Song),dcsjpf@https://www.sodocs.net/doc/3e4030327.html, (P.Jia).Expert Systems with Applications 37(2010)

1015–1022

Contents lists available at ScienceDirect

Expert Systems with Applications

journal homepag e:

https://www.sodocs.net/doc/3e4030327.html,/locate/eswa

support system based on oscillation box prediction.The basic idea of the box theory is that the stock price is supposed generally oscillates in a certain range in a period of time,which is called price box.The price will fall when it is near to the upper bound of the price box and will rise when it near to the lower bound of the box.The essence of box theory is when the price effectively breaks the upper bound or the lower bound of the oscillation box, the price will enter another oscillation box and it means the price will start an upward or downward trend.So it was the high time to buy or sell the stock(Nicolas,2007).Box theory is a powerful tool;however,its application is an experience-dependent work. The dif?culties are how to identify the price box and how to con-?rm the breakout is effective.

The proposed system is an automatic decision support system combining box theory and support vector machine algorithm. There are two modules in our trading system:oscillation box pre-diction module and trading strategy module.The support vector machines algorithm is utilized to make forecasts of the top and bottom of the oscillation box.Then trading strategy based on the box theory is constructed to make trading decisions.In the experiments,we investigate the performance of the supposed system on individual stocks with different movement patterns, test the average rate of pro?t of nearly all the stocks in S&P 500and compare with the buy-and-hold strategy.The experi-ments show a promising performance is achieved with nearly 27%average rate of return,comparing with6%by buy-and-hold strategy.

The advantage of the system lies in two aspects.First,it pro-posed a more reasonable trading tragedy which is based on the box theory but overcomes the problem of its poor performance in bearish market.Second,instead of trying to predict the true clos-ing price as other literatures did,our system forecasts the upper bound and the lower bound of the price oscillation box in a certain period of time which are less sensitive to noise than the rough clos-ing price series.This property makes the forecasts easier and more robust.

The rest of the paper is organized as follows:Section2describes the architecture and detailed design of the system.Section3intro-duces the theory of support vector machine and the layout of the SVM estimators in the system.Then the experiments and the cor-responding analysis are shown in Section4,and?nally some con-cluding remarks are drawn from Section5.2.Architecture of the stock decision support system

To simplify the problem,we assume the price with a?xed time span,i.e.n-day.Then we take the highest value H i and lowest value L i of the closing price C n-day in the future as the estimation of the two bounds in the i th day.By the de?nition of H,L,we transfer the problem of identifying the effectiveness of the box breakout to check the relationship between closing price C i and H i,L i,which consisted of the trading strategy.The basic idea of the trading sys-tem is illustrated by the Fig.1.

The architecture of the trading system shows in Fig.2.The input data include closing price and technical indicators which listed in (9).SVMmax and SVMmin are two SVM estimators which forecast the top and bottom of the price box.The model of SVMmax and SVMmin are learned incrementally from the historical price data. Based on the two forecasts,the system makes decision automati-cally according to the trading strategy.

2.1.Oscillation box prediction

In our system,we take the highest value H i and lowest value L i of the closing price C n-day in the future as the estimation of the two bounds in the i th day.

H i?maxeC i;C it1;...;C itnà1T

L i?mineC i;C it1;...;C itnà1T

Then we obtain two time series H,L corresponding to the rough price series C.The reason of the method is twofold.In the?rst,it can transform the estimation of the price box into time series prob-lem when maintains the basic meaning of price range,then we can employ the data mining method to solve it.Second,by the de?ni-tion,it can design a more robust trading strategy,which is de-scribed in the following part.Forecasting H,L is equal to make a regression for the function which is mined from the historical information.

?~H i;~L i ?fex i;x ià1;...;x iàk;y i;y ià1;...;y iàk;...T

where~H i;~L i represent the forecasts of the H i,L i,x,y are the factors related to the change of H i,L i.

The forecast models employ support vector machine algorithm. Two estimators based on SVM algorithm called SVMmax and

SVM-

Fig.1.‘"’means buy,‘;’means sell,or else do nothing.If the current close price is near to the bottom of the oscillation box and the bottom of the oscillation box is in uptrend, then buy.If the current close price is near to the top of the oscillation box and the top of the box is in downtrend,then sell.The time span n is set to30.

1016Q.Wen et al./Expert Systems with Applications37(2010)1015–1022

min are used to estimate~H i;~L i,respectively.We train the two fore-cast models by the sliding window method which is effective for time series data that is non-stationary.

2.2.Trading strategy

By the de?nition of H,L,we take the relationship between C i and H i,L i as the basic design of the trading strategy.If the current price effective breaks out the original oscillation box,it means the price will start an uptrend and form another price box,the resis-tance in the original box will become an important support in the new box.According to the de?nition of H i,L i,the current closing price C i will be close to L i at this time.The same is true for lower bound breakout.But it have been noted not all C i close to the two bounds corresponds to a box breakout.In fact every time the price oscillating to the box top and low,regardless breakout fol-lowing or not,C i is near to H i or L i and is an extreme value locally. Fortunately even when with no break,it is also a good time for short-time trading to win the price difference and we can add constraints to regulate the frequency of short-time trading.Based on the relationship between C i and H i,L i,we de?ne the trading strategy as follows.

If next trade=buy

If j C ià~L i j

C i 6r and~L i is in uptrend

If sellpriceàC i P u

then Buy,buyprice=C i

next trade=sell

else If j C ià~H i j

C i 6r and~H i is in downtrend

If C iàbuyprice P/ then Sell,sellprice=C i

next trade=buy

If buypriceàC i

buyprice P h

then Sell,sellprice=C i

next trade=buy

where r is the transaction rate which varies with the regression accuracy of SVMmax and SVMmin.When the accuracy is high,r could be smaller and vice verse.The role of r is to regulate the trade frequency.The smaller the value,the fewer the number of transac-tions.u,/are used to?lter out the false operation.h is called stop-loss rate which is commonly used in stock transaction to minimize the loss.It is very useful especially in bear market.The condition about the trend of~H i;~L i is to make sure the true trend reverse happened.3.SVMmax and SVMmin

3.1.Overview of SVM regression

The support vector machines(SVMs)algorithm is based on sta-tistical learning theory and structural risk minimization principle. It was?rst developed by Vapnik and his co-workers in the early 1990s to solve the pattern recognition(classi?cation)problem. With the introduction of loss function such as Vapnik’s-insensitive loss function,Huber’s loss function,SVMs have been extended to solve regression estimation(function approximation)problems and applied successfully in time series forecasting,non-linear modeling and optimal control problem.

The basic idea of support vector regression is to map the input space into a high-dimension space by a non-linear mapping achieved implicitly by the trick of kernel function and to do a lin-ear regression in the new feature space.Given a time series sam-ples{X i,y i}X i2R l,y i2R,i=1,2,...,n,where X i is the input feature variable,y i is the target value.SVM regression algorithm?rst maps the data to a high-dimension feature space C using a map-ping U:R l?C.Then in the high-dimension,we?nd a linear function

fex;wT?w T UexTtbe1Tin the condition of minimizing the sum of empirical risk and the complexity term k w k2which enforce?atness in feature space. Where w2C is the weight vector,b is a bias.The linear function in the high dimensional feature space corresponds to the non-linear function in the original lower dimensional feature space(Müller et al.,1997).In eàSV regression(Vapnik,1995),our goal is to?nd a function f(x)that has at most e deviation from the targets y i for all the training samples,at the same time is as?at as possible.That is to?nd(1)under the optimization problem:

minimize

1

w T w

subject to

y

i

àw T Uex iTàb6e

w T Uex iTtbày i6e

e2TConsidering the existence of data outside the e-insensitive tube

j yàew T UexTtbTj6ee3T

we introduce slack parameter d i;d?

i

to cope with the otherwise

infeasible constraints of the optimization problem(2),d i;d?

i

are de-?ned as

Q.Wen et al./Expert Systems with Applications37(2010)1015–10221017

minimize 1

2

w T wtC

X n

i?1

ed itd?

i

T

subject to

y

i

àw T Uex iTàb6etd i

w T Uex iTtbày i6etd?

i

(e4T

where C>0is a prescribed parameter to determine the trade-off be-tween the?atness of f(x)and the amount up to which deviations larger than e are tolerated.To solve this optimization problem,we construct a Lagrange function and introducing a dual set of La-grange multipliers k i;k?

i

and g i;g?i

L?1

2

w T wtC

X n

i?1

ed itd?

i

X n

i?1

k ieetd iày itw T Uex iTtbT

à

X n

i?1

k?

i

eetd?iày itw T Uex iTtbTà

X n

i?1

eg i d itg?i d?iTe5T

where k i;k?

i

;g i;g?i P0.This function has a saddle point which corre-sponds to the solution of the optimization problem.Solving(4)we get the optimal w,b and f(x):

w??

X n

i?1ek iàk?

i

TUex iT

b??

1X n

i?1

ey iàw?T Uex iT?eT;0

fexT?

X n

i?1ek iàk?

i

TUex iT UexTtb?e7T

The Lagrange multipliers k i;k?

i

in(6)are sparse.Only when x i is out-side or on the e-insensitive tube,they are non-zero.These points are called support vector.The training points in the tube are useless for f(x).The dot products in f(x)is computed in the new high dimen-sional space which is usually intractable.This problem is tackled by substituting the dot products with a kernel function which sat-is?es Mercer’s conditions.Any symmetric kernel function satisfying Mercer’s condition corresponds to a dot product in some feature space(Müller et al.,1997),then the computation of dot products in the high dimensional transforms to the simpler computation in the original low feature space.There are three commonly used ker-nel function,they are polynomial,RBF and sigmoid function.RBF kernel is most recommended,it de?ned as:

Kex i;x jT?expàjj x iàx j jj2

2g2

!

e8T

when using the RBF kernel,there are two important parameter C,g is needed to be carefully chose in the SVM algorithm.

There are some deformations of SVM algorithm such as Least Squares support vector machines(LSSVM),Linear Programming support vector machines(LPSVM)and Bayesian support vector ma-chines(BSM).For the detail refer relative literatures.

3.2.Setup of SVMmax and SVMmin

3.2.1.Input feature

In the system,we try to estimate function(1).Due to the target values related to the next n-day prices,so intuitively,the input data should at least contain the information of n-day before.The determination of n is set to30except for special note in the follow-ing experiments.The input data selected in the system totally in-clude240features.For SVMmax(SVMmin),they are:

C k;MA k;RSI k;ROC k;FastK k;SlowK k;Slow

D k;

~H

kà30e~L kà30Tk?i;ià1;...ià30

e9T

where MA k,RSI k,ROC k,FastK k,SlowK k,SlowD k are technical indica-tors computed from closing price.The computations can be found in Martin(2002).The data pre-processing is crucial to the SVM algo-rithm.All of the input data is scaling to[-1,1]by

x scaled?à1t2

xàminexT

e10T

3.2.2.Determination of parameters

When applying SVM algorithm,another important thing that needs to be considered is what kernel function is to be used.As the dynamics of?nancial time series are strongly non-linear,it is believed that using non-linear kernel functions could achieve bet-ter performance than the linear kernel.In this study,the radial ba-sis function as formula(8)is used as the kernel function of SVM. The parameter C,g are optimized by cross-validation and grid search.The training set is divided into?ve folds.One fold was ta-ken as the validation set,others taken for training.The grid point with the best accuracy of predicting is used as the value of the two parameters.

3.2.3.Training

The sliding window validation is a train and test technique which is much more suitable for time series data that is slow vary-ing or non-stationary.So the sliding window method is employed to train the two estimators.We divide the whole data set into over-lapping training-test set.The length of window is1050in which the1000data points are taken as the training set,the50data points following are taken as the test set.

4.Experiment

In order to test the feasibility of the trading system,we experi-ment in several typical stock movements such as bull market,bear market,?uctuant market and so on.The test criteria include the trade pro?t/loss and the accuracy of SVM regression.The training of SVM algorithm and the parameter n of the system is also investi-gated.All of experiments run in the MATLAB environment and all of the data are gathered from the Yahoo’s?nancial web site.The SVM algorithm employed in the experiments is the LibSVM developed by Lin,for the details about the LibSVM refer to Chang and Lin(2001).

4.1.Performance evaluation

MSE(mean squared error)and SCC(squared correlation coef?-cient)is used to measure the accuracy of SVM regression.MSE re-?ects the local?tness of the regression and SCC re?ects the global ?tness.MSE and SCC de?ned as follows

MSE?

1

N

X N

i?1

ey iày?

i

T2

SCC?

P N

i?1

ey ià y iTey ià y?

i

T

h i2

P

i?1

ey ià y iT

P

i?1

ey?

i

à y?

i

T

e11T

where y i is the actual output and y?

i

is the estimate, y i; y?

i

are their averages.

Rate of pro?t is de?ned as(12),Y0is the initial fund,Y is the?-nal return after trading strictly according to the system decision during certain period.

rate of profit?eYàY0T=Y0?100%e12TIn the trading process,suppose$1000initial fund and trade all funds/stocks at each operation.For the convenience to compare with other method the last trading decision should be sell,if not, we sell the stock at the end of the trading period.To simulate the real stock transaction,a0.5percent transaction cost for each trading is assumed.

1018Q.Wen et al./Expert Systems with Applications37(2010)1015–1022

4.2.Typical stock movement and their trade

Fig.3shows a movement in a ?uctuant and bull market and long period trade (almost 4years)with transaction rate r set to 0.05and stop-loss rate h set to 14%and u ,/set to 0,5,respectively.In the experiment,our trading system is able to pro?t up to 121.63%while the market gains about 15.3%.The MSE and SCC for SVMmax are 0.00661and 0.7109while for SVMmin they are 0.00842and 0.6677.Test data set are sampled from March 27,1990to March 7,1994.

Fig.4shows a movement in a ?uctuant and bear market and short period trade (less than 2years)with transaction rate r set to 0.12and stop-loss rate h set to 15%and u ,/set to 0,2,respec-tively.The pro?t is up to 21.04%while the stock loss is about 11.3%.The MSE and SCC for SVMmax are 0.00890and 0.8370while for SVMmin they are 0.00842and 0.6677.Test data are sampled from Jun 13,1970to Jan 11,1972.

In other word,our system can outperform buy-and-hold strat-egy in the ?uctuant market.

Movement in Fig.5is an overall bull market and 400days short trade with transaction rate r set to 0.09and stop-loss rate h set to 15%and u ,/set to 0,2,respectively.The experiment result shows our trading system achieves a perfect excellent trade with pro?t 139.5%while the stock gains about 80.6%.The result bene?ts from the good regression of the two SVM estimators.The MSE and SCC for SVMmax are 0.00662and 0.8806while for SVMmin they are 0.00776and 0.90364.Test data are sampled from Mar 17,2004to Oct 17,2005.

Movement in Fig.6is an overall bear market and 400days short trade with transaction rate r set to 0.01and stop-loss rate h set to 15%and u ,/set to 0,2,respectively.In the experiment the stock losses totally 35.2%in the period,our trading system losses only 7.6%.The MSE and SCC for SVMmax are 0.00157and 0.9921while for SVMmin they are 0.00218and 0.8336.Test data are sampled from Mar 17,2004to Oct 17,2005.4.3.Trade on S&P500

In this experiment,we evaluate the average performance on 500stocks (S&P 500components)and check whether the average pro?t outperforms the S&P 500

index.

Fig.3.Trading log(‘"’:buy,‘;’:sell)of ALCOA INC for 4

years.

Fig.4.Trading log of ALCOA INC for almost 2

years.Fig.5.Trading log of AES CP INC for 400

days.

Fig.6.Trading log of TRIBUNE INC for 400

days.

Fig.7.S&P 500index from Mar 17,2004to Oct 17,2005.

Q.Wen et al./Expert Systems with Applications 37(2010)1015–10221019

Fig.7shows the S&P 500index in 400trading days from Mar 18,2004to Oct 17,2005.The overall pro?t is 6.57%.

In the experiments,442stocks which have over 3000daily data among the S&P 500components are selected.In the SVM regres-sion training process,the cost C is set to 550and 350for SVMmax and SVMmin,respectively.The parameter g of RBF kernel function is set to 0.00002125for both estimators.The stop-loss rate h is set to 15%and u ,/set to 0,2,respectively.The average pro?t is 25.94%while the pro?t of buy-and-hold strategy is 5.77%for the 400trad-ing days,the detailed statistical result is in Table 1.That means the pro?t of the trading system is much better than the S&P 500index.For convenience of the experiments,it is needed to note the set va-lue of C ,g is not optimal for all of the stocks.

In an effort to enhance the performance,we expand the propor-tion between the number of training set and of test set from 3:1to 5:1with other parameters same for the 57stocks which underper-form the buy-and-hold strategy in uptrend and the 29stocks which loss in downtrend.The overall average pro?t rises to 27.23%.The statistical result is in Table 2.According to the Table 2,most stocks can pro?t only with 14stocks loss due to the overall downtrend during the test period and 39stocks underperform the buy-and-hold strategy due to the overall uptrend during the test period in which obviously the buy-and-hold is the best trade strategy.

Only 14stocks in the 422stocks loss due to the overall down-trend during the test period in which there is no change to pro?t.39stocks in the 422stocks underperform the buy-and-hold strat-egy due to mainly the overall uptrend during the test period in which obviously the buy-and-hold is the best trade strategy.

Due to the good performance of the box theory in bull market,all the bullish stocks investigated pro?t during the period of time and the average pro?t in bull market is as high as 39.11%.We also can ?nd even in the bear market,the system has 13.79%average pro?t when the average loss is 21.04%by buy-and-hold.That means the system in bearish market still has a good performance.4.3.1.Proportion between the number of training set and test set The experiment in Table 2raised the question of what is proper proportion between the number of training set and test set and whether the expanding of training set will enhance the perfor-

mance generally.To identify this problem,in this experiment,we expand the proportion to 5:1for all 422stocks and all of the vari-able values remain unchanged.The result shows in Table https://www.sodocs.net/doc/3e4030327.html,-pared with the experiment in Table 1,although the number of stocks underperforming buy-and-hold strategy decreases from 58to 53and the number of loss stocks decreases from 29to26slightly,the average pro?t is also decreases from 25.94%to

Table 1

Average performance of trading 422stocks for 400days.Market pattern Number of stock Less than buy-and-hold Less (%)Number of loss Loss (%)Average pro?t (%)Average pro?t of buy-and-hold (%)Bull 2245725.4400

37.6929.47Bear 19810.512914.65%12.65à21.04Total

422

58

13.12

29

6.87%

25.94

5.77

Table 2

Enhanced average performance of trading 422stocks for 400days.Market pattern Number of stock Less than buy-and-hold Less (%)Number of loss Loss (%)Average pro?t (%)Average pro?t of buy-and-hold (%)Bull 2243917.41%00

39.1129.47Bear 19800

147.07%13.79à21.04Total

422

39

9.24%

14

3.32

27.23%

5.77

Table 3

Average performance of trading 422stocks for 400days with extended training set.Market pattern Number of stock Less than buy-and-hold Less (%)Number of loss Loss (%)Average pro?t (%)Average pro?t of buy-and-hold (%)Bull 2245122.7700

36.2929.47Bear 1982 1.012613.13%10.33à21.04Total

422

53

12.56

26

8.53%

24.11

5.77

Fig.8.Trading log of AES CP INC for 400days from Mar 17,2004to Oct 17,2005.with transaction rate r set to 0.25.Top:There is 1400data training points,the pro?t is 23.92%.Bottom:the training set extend to 2492points,the pro?t is 14.02%.

1020Q.Wen et al./Expert Systems with Applications 37(2010)1015–1022

24.11%.We also?nd from the experiment that the performances improve only when the expanded training set have the same mov-ing pattern with the test set,otherwise,the expanding of training set often leads to deterioration of performance.An example shows in Fig.8.

4.3.2.Choice of n

The width of the oscillation box n is another parameter in this trading system need to be identi?ed.Intuitively,the n smaller, the higher the accuracy of the two estimators SVMmax and SVMmin will be got,but the trade decision may be less optimal due to the forecasts concerning less days ahead and vice verse.

In order to identify n,this experiment is carried on50stocks (the alphabetically?rst50stock in S&P500).The average pro?t is investigated by varying n.the n changes from3to50.The result shows in Table4.*/*means to predict the upper/lower bound of the oscillation box*days ahead using the information*days be-fore.Note that the information of stock in a day includes eight fea-tures as(9).

From the result,we can?nd the best performance achieved in n=8and30.Further studying the movements of the50stocks, the performance is better with n=8when the stocks are dramati-cally?uctuant during the test period,the better performance is achieved with n=30when the stocks change gently.

According the experiment,n set to8is recommended when the market is in dramatic?uctuation,otherwise n set to30is recommended.

4.4.Trade on individual stock

This experiment is carried on two commonly selected stocks Microsoft and IBM in many literatures.Number of transactions is the number of the buy-and-sell operation.

Table5shows the performance of trading Microsoft.The time span n is set to30and the stop-loss rate h is set to10%and all of other parameters have the same values as the experiment in Table 1.The MSE and SCC for SVMmax are0.00890and0.8370while for SVMmin they are0.00842and0.6677.When transaction rate r is set to0.025the system achieves the best pro?t with74.00%.The trade log shows in Fig.9.

Table6lists the performance of trading IBM.The time span n is set to30,the cost C is set to1000and2000for SVMmax and SVM-min,respectively while g is set to0.00002125for both estimator. The stop-loss rate h is set to15%and u,/set to0,2,respectively. The MSE and SCC for SVMmax are0.00016and0.9111while for SVMmin they are0.00020and0.8706.When transaction rate r is set to0.015the system achieves the best pro?t with53.31%.The trade log shows in Fig.10.

5.Conclusion and future work

In this paper,we present a novel implementation of intelligent stock decision support system based on oscillation box prediction. Two estimator based on SVM regression algorithm is used to fore-cast the upper and lower bound of the oscillation box respectively. We also investigated the choice of the width of the box and the training of the SVM algorithm in?nancial time series.In order to examine the feasibility,we test the system on typical stock move-ments and S&P500components and typical individual stocks.The experiments show a promising performance and dramatically out-perform buy-and-hold strategy.

The prediction algorithms with a single algorithm are often fragile because of the complexity of the stock movement,and there is a wide acceptance of bene?t of the synergy effect.In the system,

Table4

Average performance of trading50stocks by varying n.

3/35/58/810/1015/1520/2030/3040/4050/50

Average pro?t(%)35.539.4940.7939.8334.3535.8637.8837.0134.89 number of transaction9.88.78.17.77.38.48.87.97.7 Average SCC of SVMmax0.9690.9450.9140.8880.8430.7920.7060.6230.562 Average SCC of SVMmin0.9590.9250.8810.8520.7860.7320.6090.5230.462

Table5

Performance of trading microsoft.

r0.0050.010.0150.020.0250.030.0350.04Buy-and-hold Number of transactions457888661

Pro?t(%)à2.7719.0353.3773.9474.0072.1729.3715.41

5.16

Fig.9.Trading log of Microsoft for4years from Feb12,2004to Feb12,2008with

trade ratio r set to0.025and pro?t74.00%.The pro?t is5.16%by the buy-and-hold

strategy.

Table6

Performance of trading IBM.

r0.0050.010.0150.020.0250.030.0350.04Buy-and-hold

Number of transactions22344610131

Pro?t(%)10.1047.3853.3120.8324.33 5.9715.33à13.637.28

Q.Wen et al./Expert Systems with Applications37(2010)1015–10221021

the accuracy of the forecasts for the upper and lower bound of the oscillation box is also the bottleneck of the performance.So the fu-ture work includes building more robust estimators to improve the accuracy of the forecasts by combining other soft computing tech-niques together.References

Baba,N.,Inoue,N.,Yanjun,Y.(2002).Utilization of soft computing techniques for

constructing reliable decision support system for dealing stocks.In IJCNN’02:

Proceedings of the 2002international joint conference on neural networks,Honolulu,Hawaii .

Bao,Y.-K.(2005).Forecasting stock composite index by fuzzy support vector

machines regression.In Proceedings of the fourth international conference on machine learning and cybernetics,Guangzhou,18–21August 2005.

Bao, D.,&Yang,Z.(2008).Intelligent stock trading system by turning point

con?rming and probabilistic reasoning.Expert Systems with Applications,34(1),620–627.

Cao,L.J.,&Francis, E.H.(2003).Tay,support vector machine with adaptive

parameters in ?nancial time series forecasting.IEEE Transactions on Neural Networks,14(6).

Chang, C.-C.,Lin, C.-J.(2001).LIBSVM:A library for support vector machines.

Software available at .

Chen,A.S.,&Leung,M.T.(2004).Regression neural network for error correction in

foreign exchange forecasting and https://www.sodocs.net/doc/3e4030327.html,puters and Operations Research,31,1049–1068.

Grudnitski,G.,&Osburn,L.(1993).Forecasting S&P and gold futures prices:an

application of neural networks.Journal of Futures Market,13,631–643.

Hassan,M.R.,Nath,B.,(2005).Stock market forecasting using hidden Markov

model:A new approach.In Proceedings of the ?fth international conference on intelligent systems design and applications (pp.192–196).

Kwon,Y.-K.,&Moon, B.-R.(2007).A hybrid neurogenetic approach for stock

forecasting.IEEE Transaction on Neural Networks,18(3).

Lee,J.W.(2001).Stock price prediction using reinforcement learning.In IEEE

international joint conference on neural networks (pp.690–695).

Li,J.-B.,Chen,A.-P.(2006).Re?ned group learning based on XCS and neural network

in intelligent ?nancial decision support system.In Proceedings of the sixth international conference on intelligent systems design and applications (ISDA’06).Lin,X.,Yang,Z.,Song,Y.,Washio T.,et al.(2008).The application of echo state

network in stock data mining.In PAKDD 2008:Vol.5012.LNAI (pp.932–937).Martin,P.(2002).Technical analysis explained (4th ed.).Paperback.McGraw-Hill

Company.ISBN0071226699.

Müller,K.-R.,Smola,A.,R?tsch,G.,Sch?lkopf,B.,Kohlmorgen,J.,Vapnik,V.(1997).

Predicting time series with support vector machines.In Proceedings,international conference on arti?cial neural networks.Lecture notes in computer science (p.999).Springer.

Nicolas,D.(2007).How I made two million dollars in the stock market .BN Publishing.Vapnik,V.N.(1995).The nature of statistical learning theory .New York:

Springer.

Fig.10.Trading log of IBM for 4years from Feb 12,2004to Feb 12,2008with trade ratio r set to 0.015and pro?t 53.31%.The pro?t is 7.28%by the buy-and-hold strategy.

1022Q.Wen et al./Expert Systems with Applications 37(2010)1015–1022

(完整word版)支持向量机(SVM)原理及应用概述分析

支持向量机(SVM )原理及应用 一、SVM 的产生与发展 自1995年Vapnik (瓦普尼克)在统计学习理论的基础上提出SVM 作为模式识别的新方法之后,SVM 一直倍受关注。同年,Vapnik 和Cortes 提出软间隔(soft margin)SVM ,通过引进松弛变量i ξ度量数据i x 的误分类(分类出现错误时i ξ大于0),同时在目标函数中增加一个分量用来惩罚非零松弛变量(即代价函数),SVM 的寻优过程即是大的分隔间距和小的误差补偿之间的平衡过程;1996年,Vapnik 等人又提出支持向量回归 (Support Vector Regression ,SVR)的方法用于解决拟合问题。SVR 同SVM 的出发点都是寻找最优超平面(注:一维空间为点;二维空间为线;三维空间为面;高维空间为超平面。),但SVR 的目的不是找到两种数据的分割平面,而是找到能准确预测数据分布的平面,两者最终都转换为最优化问题的求解;1998年,Weston 等人根据SVM 原理提出了用于解决多类分类的SVM 方法(Multi-Class Support Vector Machines ,Multi-SVM),通过将多类分类转化成二类分类,将SVM 应用于多分类问题的判断:此外,在SVM 算法的基本框架下,研究者针对不同的方面提出了很多相关的改进算法。例如,Suykens 提出的最小二乘支持向量机 (Least Square Support Vector Machine ,LS —SVM)算法,Joachims 等人提出的SVM-1ight ,张学工提出的中心支持向量机 (Central Support Vector Machine ,CSVM),Scholkoph 和Smola 基于二次规划提出的v-SVM 等。此后,台湾大学林智仁(Lin Chih-Jen)教授等对SVM 的典型应用进行总结,并设计开发出较为完善的SVM 工具包,也就是LIBSVM(A Library for Support Vector Machines)。LIBSVM 是一个通用的SVM 软件包,可以解决分类、回归以及分布估计等问题。 二、支持向量机原理 SVM 方法是20世纪90年代初Vapnik 等人根据统计学习理论提出的一种新的机器学习方法,它以结构风险最小化原则为理论基础,通过适当地选择函数子集及该子集中的判别函数,使学习机器的实际风险达到最小,保证了通过有限训练样本得到的小误差分类器,对独立测试集的测试误差仍然较小。 支持向量机的基本思想:首先,在线性可分情况下,在原空间寻找两类样本的最优分类超平面。在线性不可分的情况下,加入了松弛变量进行分析,通过使用非线性映射将低维输

支持向量机分类器

支持向量机分类器 1 支持向量机的提出与发展 支持向量机( SVM, support vector machine )是数据挖掘中的一项新技术,是借助于最优化方法来解决机器学习问题的新工具,最初由V.Vapnik 等人在1995年首先提出,近几年来在其理论研究和算法实现等方面都取得了很大的进展,开始成为克服“维数灾难”和过学习等困难的强有力的手段,它的理论基础和实现途径的基本框架都已形成。 根据Vapnik & Chervonenkis的统计学习理论 ,如果数据服从某个(固定但未知的)分布,要使机器的实际输出与理想输出之间的偏差尽可能小,则机器应当遵循结构风险最小化 ( SRM,structural risk minimization)原则,而不是经验风险最小化原则,通俗地说就是应当使错误概率的上界最小化。SVM正是这一理论的具体实现。与传统的人工神经网络相比, 它不仅结构简单,而且泛化( generalization)能力明显提高。 2 问题描述 2.1问题引入 假设有分布在Rd空间中的数据,我们希望能够在该空间上找出一个超平面(Hyper-pan),将这一数据分成两类。属于这一类的数据均在超平面的同侧,而属于另一类的数据均在超平面的另一侧。如下图。 比较上图,我们可以发现左图所找出的超平面(虚线),其两平行且与两类数据相切的超平面(实线)之间的距离较近,而右图则具有较大的间隔。而由于我们希望可以找出将两类数据分得较开的超平面,因此右图所找出的是比较好的超平面。 可以将问题简述如下: 设训练的样本输入为xi,i=1,…,l,对应的期望输出为yi∈{+1,-1},其中+1和-1分别代表两类的类别标识,假定分类面方程为ω﹒x+b=0。为使分类面对所有样本正确分类并且具备分类间隔,就要求它满足以下约束条件: 它追求的不仅仅是得到一个能将两类样本分开的分类面,而是要得到一个最优的分类面。 2.2 问题的数学抽象 将上述问题抽象为: 根据给定的训练集

基于机器学习的股票分析与预测模型研究

金融观察?一 基于机器学习的股票分析与预测模型研究① 姚雨琪 摘一要:近年来?随着全球经济与股市的快速发展?股票投资成为人们最常用的理财方式之一?本文研究的主要目标是利用机器学习技术?应用Python编程语言构建股票预测模型?对我国股票市场进行分析与预测?采用SVM与DTW构建股票市场的分析和预测模型?并通过Python编程进行算法实现? 本文对获取到的股票数据进行简单策略分析?选取盘中策略作为之后模型评估的基准线?分别选取上证指数二鸿达兴业股票二鼎汉股票数据利用已构建的支持向量机和时间动态扭曲模型在Python平台上进行预测分析?结果表明?对于上证指数而言?支持向量机预测下逆向策略更优?对于鸿达兴业股票和鼎汉股票而言?支持向量机预测下正向策略更优?基于时间动态扭曲算法的预测方法对于特定的股票有较高的精度和可信度?研究结论表明将机器学习运用于股票分析与预测可以提高股票价格信息预测的效率?保证对海量数据的处理效率?机器学习过程可以不断进行优化模型?使得预测的可信度和精度不断提高?机器学习技术在股票分析方面有很高的研究价值? 关键词:机器学习?股票预测?Python?SVM?DTW 中图分类号:F830.91一一一一一一文献标识码:A一一一一一一文章编号:1008-4428(2019)02-0123-02 一一一二引言 国外股票市场的股票分析预测开始得很早?研究者们将各种数学理论二数据挖掘技术等应用到股票分析软件中?并通过对历史交易数据的研究?从而得到股票的走势规律?近年来?由于现实中工作与研究的需要?机器学习的研究与应用在国内外越来越重视?机器学习可以在运用过程中依据新的数据不断学习优化?完善预测模型?将机器学习应用于股票市场的预测?从股票的历史数据中挖掘出隐藏在数据中的重要信息?这样既能够为股民们对股价预测研究提供理论支撑?又能够为公司的领导层提供决策支持?基于此?本文选择机器学习在股票分析中的应用作为研究方向?在机器学习及股票分析相关理论基础上?使用Python开发工具?并分别运用支持向量回归及时间动态扭曲进行预测? 二二相关技术与理论 (一)机器学习 机器学习是融合多领域技术的交叉学科?主要包括概率论与数理统计二微积分二线性代数二算法设计等多门学科?通过计算机相关技术自动 学习 实现人工智能?(二)股票分析方法 1.基本面分析 基本面分析指的是在分析股票市场供应和需求关系的相关因素(如宏观经济二政策导向二财务状况以及经营环境等)基础上确定股票的实际价格?从而预测股票价格的趋势?2.技术面分析 技术面分析指的是对股票图样趋势来分析和研究?来判断价格的走势? (三)基于Python的经典机器学习模型 1.支持向量机(SVM) 该模型最初用于分类?其最终目标是引入回归估计?建立回归估计函数G(x)?其中回归值与目标值之间的差值小于μ?同时保证该函数的VC维度最小?线性或非线性函数G(x)的回归问题可以转化为二次规划问题?并且获得的最优解是唯一的? 2.动态时间扭曲(DTW) 这是衡量时间序列之间的相似性的方法?并可以用在语音识别领域以判断两段声音是否表达了同一个意思?三二股票预测模型的构建 (一)确定初始指标 1.基于支持向量机确定指标 施燕杰(2005)利用支持向量机进行股票分析与预测?在多次反复尝试基础上提出了一系列的指标作为预测模型的输入向量?该指标能够有效地预测未来股价波动情况?本文在结合自身研究的基础上?对以上施燕杰提出的指标进行改进?在原有的指标基础上添加7日平均开盘价和7日平均收盘价?去除了成交额保留了成交量?最终建立如表1所示的20个初选指标? 表1一初选指标 变量X1X2X3X4X5X6X7X8X9X10含义 今日 开盘价 昨日 开盘价 前日 开盘价 7日平均 开盘价 今日 最高价 昨日 最高价 前日 最高价 7日平均 最高价 今日 最低价 昨日 最低价变量X11X12X13X14X15X16X17X18X19X20含义 前日 最低价 7日平均 最低价 今日 收盘价 昨日 收盘价 前日 收盘价 7日平均 收盘价 今日 成交量 昨日 成交量 前日 成交量 7日平均 成交量一一本文主要是进行股票分析与预测?因此在综合考虑各个 价格指标的基础上?本文选择选定时间段的下一日收盘价作为模型的输出向量? 2.基于动态时间扭曲确定指标 根据往常研究经验?我们将时间序列数据分成不同的期间?每个期间长度为5日?以每个时间段相邻每日收盘价涨跌率变化趋势为初始指标?选择时间序列期间下一日的收盘价与期间内最后一日收盘价涨跌率作为模型的输出向量?(二)选择样本 1.实验对象 本文在分别在主板市场二中小板市场和创业板市场中采取随机抽样的方法各随机选择一只股票数据作为研究对象?分别是上证指数二鸿达兴业股票二鼎汉股票? 2.样本规模 我们选取了2011年至2017年间上证指数1550条数据?2015年至2017年的鸿达兴业股票532条数据二鼎汉股票572 321 ①基金项目:江西财经大学第十三届科研课题立项?编号xskt18345?

支持向量机的实现

模式识别课程大作业报告——支持向量机(SVM)的实现 姓名: 学号: 专业: 任课教师: 研究生导师: 内容摘要

支持向量机是一种十分经典的分类方法,它不仅是模式识别学科中的重要内容,而且在图像处理领域中得到了广泛应用。现在,很多图像检索、图像分类算法的实现都以支持向量机为基础。本次大作业的内容以开源计算机视觉库OpenCV为基础,编程实现支持向量机分类器,并对标准数据集进行测试,分别计算出训练样本的识别率和测试样本的识别率。 本报告的组织结构主要分为3大部分。第一部分简述了支持向量机的原理;第二部分介绍了如何利用OpenCV来实现支持向量机分类器;第三部分给出在标准数据集上的测试结果。 一、支持向量机原理概述

在高维空间中的分类问题实际上是寻找一个超平面,将两类样本分开,这个超平面就叫做分类面。两类样本中离分类面最近的样本到分类面的距离称为分类间隔。最优超平面指的是分类间隔最大的超平面。支持向量机实质上提供了一种利用最优超平面进行分类的方法。由最优分类面可以确定两个与其平行的边界超平面。通过拉格朗日法求解最优分类面,最终可以得出结论:实际决定最优分类面位置的只是那些离分类面最近的样本。这些样本就被称为支持向量,它们可能只是训练样本中很少的一部分。支持向量如图1所示。 图1 图1中,H是最优分类面,H1和H2别是两个边界超平面。实心样本就是支持向量。由于最优超平面完全是由这些支持向量决定的,所以这种方法被称作支持向量机(SVM)。 以上是线性可分的情况,对于线性不可分问题,可以在错分样本上增加一个惩罚因子来干预最优分类面的确定。这样一来,最优分类面不仅由离分类面最近的样本决定,还要由错分的样本决定。这种情况下的支持向量就由两部分组成:一部分是边界支持向量;另一部分是错分支持向量。 对于非线性的分类问题,可以通过特征变换将非线性问题转化为新空间中的线性问题。但是这样做的代价是会造成样本维数增加,进而导致计算量急剧增加,这就是所谓的“维度灾难”。为了避免高维空间中的计算,可以引入核函数的概念。这样一来,无论变换后空间的维数有多高,这个新空间中的线性支持向量机求解都可以在原空间通过核函数来进行。常用的核函数有多项式核、高斯核(径向基核)、Sigmoid函数。 二、支持向量机的实现 OpenCV是开源计算机视觉库,它在图像处理领域得到了广泛应用。OpenCV 中包含许多计算机视觉领域的经典算法,其中的机器学习代码部分就包含支持向量机的相关内容。OpenCV中比较经典的机器学习示例是“手写字母分类”。OpenCV 中给出了用支持向量机实现该示例的代码。本次大作业的任务是研究OpenCV中的支持向量机代码,然后将其改写为适用于所有数据库的通用程序,并用标准数据集对算法进行测试。本实验中使用的OpenCV版本是,实验平台为Visual

支持向量机在股票分析中的应用

1、问题的背景股票市场具有高收益与高风险并存的特性,人们一直都希望能够掌握其运行规律,并进行分析与预测。但是由于股票市场受到很多市场因素和非市场的影响,而且这些因素之间又有相互作用,因此要想建立一个模型来描述其内部相互作用的机理是非常困难的。从而这也成为证券分析研究领域的一个难题。股票价格的变化是非线性和时变的,支持向量机在股票分析中的应用 杨明海 信阳师范学院数学科学学院 464000 因此这些时间序列数据都是带有噪声的不稳定随机数据,从而使得用传统的统计方法来研究股票市场的运行规律已经不尽如人意。随着计算机科学的发展,把机器学习方法用在金融工程领域已经取得了很大的进展[1]。 当前很多模型的建立都要假定股票价 格序列具有某些非线性特征,这些假设或多或少的带有一些主观性。对于这类数据的预测很多研究者采用神经网络等方法来 做,但神经网络方法有着难以克服的一些 缺点,其网络结构需要事先指定或应用启 发式算法在训练过程中修正,这些启发式算法很难保证网络结构是最优的。另外神经网络易过学习和陷入局部最优的缺点也极大地限制其在实际中的应用。而支持向量机(SVM )它克服了神经网络的一些缺 点,如过学习,维数灾难,易陷入局部最优 等。而且支持向量机在实现金融时间序列 的预测[2]方面已有了一些探索性的研究,并在应用中取得了不错的表现。 2、支持向量机 20世纪60年代末,V a p n i k 和 Cervonenkis 建立了现代意义上的统计学习 理论[3],即是目前人们所称的V C 维理 论。1979年,在统计学习理论的VC 维理论 和结构风险最小化(SRM)准则的基础上, Vapnik 等人提出了一种新的机器学习算法 ——支持向量机(SVM)方法。支持向量机 方法根据有限样本的信息在模型的复杂性 (基于VC 维,即对特定训练集的训练精度)和学习能力(即由训练出的机器对测试数 据的识别能力)之间寻求最佳折衷,期望获 得最好的推广能力。大量的研究表明,作为 一种解决模式识别问题和非线性函数的回 归估计、预测等问题的新技术,SVM 无论 在模型拟合效果还是模型推广能力方面都 表现出了良好的性质。因此,它成为目前机 器学习领域的一个热门研究课题。当前对 SVM 的研究集中在训练算法的设计和应用 上。 支持向量机可以归结为解决一个二次规划问题(Quadratic Programming, QP ) [4]: 给定输入空间中训练样本:(1) SVM 是要得到下面的决策函数: (2)其中k(x i ,x j )是核函数,是每个样本对应的Lagrange 乘子,b 是阈值,是下面QP 问题的解:其中(核函数矩阵)是一个N ×N 的半正定矩阵,C 是正则化参数。 由于其坚实的理论基础,良好的泛化性能,简洁的数学形式,直观的几何解释 等特点,它在许多实际问题的应用中取得 了成功。本文用支持向量机方法对中国A 股市场深沪两市的部分部分股票进行分类 研究。 3、数值计算结果及分析3.1 实验数据由股票分析软件广发证券(至强版)下载了深沪股市几个交易日的数据,预处理如下: DATA1:选取2009年1月6号的1296只股票,以量比,换手率,内外比为因子,将涨幅大于一个百分点的股票归为+1类,跌幅大于一个百分点的归为-1类。将数据随机化后,选取1000个进行训练,296个进行测试。DATA2:选取2008年12月29号沪深两市共1192只股票,以每笔换手率,涨速,量比,总换手率,内外比,振幅为因子,将涨幅大于1.5个百分点的股票归

支持向量机原理及应用(DOC)

支持向量机简介 摘要:支持向量机方法是建立在统计学习理论的VC 维理论和结构风险最小原理基础上的,根据有限的样本信息在模型的复杂性(即对特定训练样本的学习精度)和学习能力(即无错误地识别任意样本的能力)之间寻求最佳折衷,以求获得最好的推广能力 。我们通常希望分类的过程是一个机器学习的过程。这些数据点是n 维实空间中的点。我们希望能够把这些点通过一个n-1维的超平面分开。通常这个被称为线性分类器。有很多分类器都符合这个要求。但是我们还希望找到分类最佳的平面,即使得属于两个不同类的数据点间隔最大的那个面,该面亦称为最大间隔超平面。如果我们能够找到这个面,那么这个分类器就称为最大间隔分类器。 关键字:VC 理论 结构风险最小原则 学习能力 1、SVM 的产生与发展 自1995年Vapnik 在统计学习理论的基础上提出SVM 作为模式识别的新方法之后,SVM 一直倍受关注。同年,Vapnik 和Cortes 提出软间隔(soft margin)SVM ,通过引进松弛变量i ξ度量数据i x 的误分类(分类出现错误时i ξ大于0),同时在目标函数中增加一个分量用来惩罚非零松弛变量(即代价函数),SVM 的寻优过程即是大的分隔间距和小的误差补偿之间的平衡过程;1996年,Vapnik 等人又提出支持向量回归 (Support Vector Regression ,SVR)的方法用于解决拟合问题。SVR 同SVM 的出发点都是寻找最优超平面,但SVR 的目的不是找到两种数据的分割平面,而是找到能准确预测数据分布的平面,两者最终都转换为最优化问题的求解;1998年,Weston 等人根据SVM 原理提出了用于解

支持向量机数据分类预测

支持向量机数据分类预测 一、题目——意大利葡萄酒种类识别 Wine数据来源为UCI数据库,记录同一区域三种品种葡萄酒的化学成分,数据有178个样本,每个样本含有13个特征分量。50%做为训练集,50%做为测试集。 二、模型建立 模型的建立首先需要从原始数据里把训练集和测试集提取出来,然后进行一定的预处理,必要时进行特征提取,之后用训练集对SVM进行训练,再用得到的模型来预测试集的分类。 三、Matlab实现 3.1 选定训练集和测试集 在178个样本集中,将每个类分成两组,重新组合数据,一部分作为训练集,一部分作为测试集。 % 载入测试数据wine,其中包含的数据为classnumber = 3,wine:178*13的矩阵,wine_labes:178*1的列向量 load chapter12_wine.mat; % 选定训练集和测试集 % 将第一类的1-30,第二类的60-95,第三类的131-153做为训练集 train_wine = [wine(1:30,:);wine(60:95,:);wine(131:153,:)]; % 相应的训练集的标签也要分离出来 train_wine_labels = [wine_labels(1:30);wine_labels(60:95);wine_labels(131:153)]; % 将第一类的31-59,第二类的96-130,第三类的154-178做为测试集 test_wine = [wine(31:59,:);wine(96:130,:);wine(154:178,:)]; % 相应的测试集的标签也要分离出来 test_wine_labels = [wine_labels(31:59);wine_labels(96:130);wine_labels(154:178)]; 3.2数据预处理 对数据进行归一化: %% 数据预处理 % 数据预处理,将训练集和测试集归一化到[0,1]区间 [mtrain,ntrain] = size(train_wine); [mtest,ntest] = size(test_wine); dataset = [train_wine;test_wine]; % mapminmax为MATLAB自带的归一化函数 [dataset_scale,ps] = mapminmax(dataset',0,1); dataset_scale = dataset_scale';

用于分类的支持向量机

文章编号:100228743(2004)0320075204 用于分类的支持向量机 黄发良,钟 智Ξ (1.广西师范大学计算机系,广西桂林541000;  2.广西师范学院数学与计算机科学系,广西南宁530001) 摘 要:支持向量机是20世纪90年代中期发展起来的机器学习技术,建立在结构风险最小化原理之上的支持向量机以其独有的优点吸引着广大研究者,该文着重于用于分类的支持向量机,对其基本原理与主要的训练算法进行介绍,并对其用途作了一定的探索. 关键词:支持向量机;机器学习;分类 中图分类号:TP181 文献标识码:A 支持向量机S VM (Support Vector Machine )是AT&T Bell 实验室的V.Vapnik 提出的针对分类和回归问题的统计学习理论.由于S VM 方法具有许多引人注目的优点和有前途的实验性能,越来越受重视,该技术已成为机器学习研究领域中的热点,并取得很理想的效果,如人脸识别、手写体数字识别和网页分类等. S VM 的主要思想可以概括为两点:(1)它是针对线性可分情况进行分析,对于线性不可分的情况,通过使用非线性映射算法将低维输入空间线性不可分的样本转化为高维特征空间使其线性可分,从而使得高维特征空间采用线性算法对样本的非线性特征进行线性分析成为可能;(2)它基于结构风险最小化理论之上在特征空间中建构最优分割超平面,使得学习器得到全局最优化,并且在整个样本空间的期望风险以某个概率满足一定上界. 1 基本原理 支持向量机理论最初来源于数据分类问题的处理,S VM 就是要寻找一个满足要求的分割平面,使训练集中的点距离该平面尽可能地远,即寻求一个分割平面使其两侧的margin 尽可能最大. 设输入模式集合{x i }∈R n 由两类点组成,如果x i 属于第1类,则y i =1,如果x i 属于第2类,则y i =-1,那么有训练样本集合{x i ,y i },i =1,2,3,…,n ,支持向量机的目标就是要根据结构风险最小化原理,构造一个目标函数将两类模式尽可能地区分开来,通常分为两类情况来讨论,(1)线性可分,(2)线性不可分. 1.1 线性可分情况 在线性可分的情况下,就会存在一个超平面使得训练样本完全分开,该超平面可描述为: w ?x +b =0(1) 其中,“?”是点积,w 是n 维向量,b 为偏移量. 最优超平面是使得每一类数据与超平面距离最近的向量与超平面之间的距离最大的这样的平面.最优超平面可以通过解下面的二次优化问题来获得: min <(w )= 12‖w ‖2(2) Ξ收稿日期:2004202206作者简介:黄发良(1975-),男,湖南永州人,硕士研究生;研究方向:数据挖掘、web 信息检索. 2004年9月 广西师范学院学报(自然科学版)Sep.2004 第21卷第3期 Journal of G u angxi T eachers Education U niversity(N atural Science Edition) V ol.21N o.3

基于支持向量机的分类方法

基于支持向量机的分类方法 摘要:本文首先概述了支持向量机的相关理论,引出了支持向量机的基本模型。当训练集的两类样本点集重合区域很大时,线性支持向量分类机就不适用了,由此介绍了核函数相关概念。然后进行了核函数的实验仿真,并将支持向量机应用于实例肿瘤诊断,建立了相应的支持向量机模型,从而对测试集进行分类。最后提出了一种支持向量机的改进算法,即根据类向心度对复杂的训练样本进行预删减。 1、支持向量机 给定训练样本集1122{[,],[,], ,[,]}()l l l T a y a y a y Y =∈Ω?L ,其中n i a R ∈Ω=,Ω是输入空间,每一个点i a 由n 个属性特征组成,{1,1},1,,i y Y i l ∈=-=L 。分类 就是在基于训练集在样本空间中找到一个划分超平面,将不同的类别分开,划分超平面可通过线性方程来描述: 0T a b ω+= 其中12(;;;)d ωωωω=K 是法向量,决定了超平面的方向,b 是位移项,决定 了超平面与原点之间的距离。样本空间中任意点到超平面的距离为|| |||| T a b r ωω+=。 支持向量、间隔: 假设超平面能将训练样本正确分类,即对于[,]i i a y T ∈,若1i y =+,则有 0T i a b ω+>,若1i y =-,则有0T i a b ω+<。则有距离超平面最近的几个训练样本点使得 11 11 T i i T i i a b y a b y ωω?+≥+=+?+≤-=-? 中的等号成立,这几个训练样本点被称为支持向量;两个异类支持向量到超平面 的距离之和2 |||| r ω=被称为间隔。 支持向量机基本模型: 找到具有最大间隔的划分超平面,即 ,2max ||||..()1,1,2,...,b T i i s t y a b i m ωωω+≥= 这等价于 2 ,||||min 2..()1,1,2,...,b T i i s t y a b i m ωωω+≥= 这就是支持向量机(SVM )的基本模型。 支持向量机问题的特点是目标函数2 ||||2 ω是ω的凸函数,并且约束条件都是 线性的。

支持向量机(SVM)原理及应用概述

支持向量机(SVM)原理及应用 一、SVM得产生与发展 自1995年Vapnik(瓦普尼克)在统计学习理论得基础上提出SVM作为模式识别得新方法之后,SVM一直倍受关注。同年,Vapnik与Cortes提出软间隔(soft margin)SVM,通过引进松弛变量度量数据得误分类(分类出现错误时大于0),同时在目标函数中增加一个分量用来惩罚非零松弛变量(即代价函数),SVM得寻优过程即就是大得分隔间距与小得误差补偿之间得平衡过程;1996年,Vapnik等人又提出支持向量回归 (Support Vector Regression,SVR)得方法用于解决拟合问题。SVR同SVM得出发点都就是寻找最优超平面(注:一维空间为点;二维空间为线;三维空间为面;高维空间为超平面。),但SVR得目得不就是找到两种数据得分割平面,而就是找到能准确预测数据分布得平面,两者最终都转换为最优化问题得求解;1998年,Weston等人根据SVM原理提出了用于解决多类分类得SVM方法(MultiClass Support Vector Machines,MultiSVM),通过将多类分类转化成二类分类,将SVM应用于多分类问题得判断:此外,在SVM算法得基本框架下,研究者针对不同得方面提出了很多相关得改进算法。例如,Suykens 提出得最小二乘支持向量机(Least Square Support Vector Machine,LS—SVM)算法,Joachims等人提出得SVM1ight,张学工提出得中心支持向量机 (Central Support Vector Machine,CSVM),Scholkoph与Smola基于二次规划提出得vSVM等。此后,台湾大学林智仁(Lin ChihJen)教授等对SVM得典型应用进行总结,并设计开发出较为完善得SVM工具包,也就就是LIBSVM(A Library for Support Vector Machines)。LIBSVM就是一个通用得SVM软件包,可以解决分类、回归以及分布估计等问题。 二、支持向量机原理 SVM方法就是20世纪90年代初Vapnik等人根据统计学习理论提出得一种新得机器学习方法,它以结构风险最小化原则为理论基础,通过适当地选择函数子集及该子集中得判别函数, 使学习机器得实际风险达到最小,保证了通过有限训练样本得到得小误差分类器,对独立测试集得测试误差仍然较小。 支持向量机得基本思想:首先,在线性可分情况下,在原空间寻找两类样本得最优分类超平面。在线性不可分得情况下,加入了松弛变量进行分析,通过使用非线性映射将低维输入空

支持向量机SVM分类算法

支持向量机SVM分类算法 SVM的简介 支持向量机(Support Vector Machine)是Cortes和Vapnik于1995年首先提出的,它在解决小样本、非线性及高维模式识别中表现出许多特有的优势,并能够推广应用到函数拟合等其他机器学习问题中[10]。 支持向量机方法是建立在统计学习理论的VC 维理论和结构风险最小原理基础上的,根据有限的样本信息在模型的复杂性(即对特定训练样本的学习精度,Accuracy)和学习能力(即无错误地识别任意样本的能力)之间寻求最佳折衷,以期获得最好的推广能力[14](或称泛化能力)。 以上是经常被有关SVM 的学术文献引用的介绍,我来逐一分解并解释一下。 Vapnik是统计机器学习的大牛,这想必都不用说,他出版的《Statistical Learning Theory》是一本完整阐述统计机器学习思想的名著。在该书中详细的论证了统计机器学习之所以区别于传统机器学习的本质,就在于统计机器学习能够精确的给出学习效果,能够解答需要的样本数等等一系列问题。与统计机器学习的精密思维相比,传统的机器学习基本上属于摸着石头过河,用传统的机器学习方法构造分类系统完全成了一种技巧,一个人做的结果可能很好,另一个人差不多的方法做出来却很差,缺乏指导和原则。所谓VC维是对函数类的一种度量,可以简单的理解为问题的复杂程度,VC维越高,一个问题就越复杂。正是因为SVM关注的是VC维,后面我们可以看到,SVM解决问题的时候,和样本的维数是无关的(甚至样本是上万维的都可以,这使得SVM很适合用来解决文本分类的问题,当然,有这样的能力也因为引入了核函数)。 结构风险最小听上去文绉绉,其实说的也无非是下面这回事。 机器学习本质上就是一种对问题真实模型的逼近(我们选择一个我们认为比较好的近似模型,这个近似模型就叫做一个假设),但毫无疑问,真实模型一定是不知道的(如果知道了,我们干吗还要机器学习?直接用真实模型解决问题不就可以了?对吧,哈哈)既然真实模型不知道,那么我们选择的假设与问题真实解之间究竟有多大差距,我们就没法得知。比如说我们认为宇宙诞生于150亿年前的一场大爆炸,这个假设能够描述很多我们观察到的现象,但它与真实的宇宙模型之间还相差多少?谁也说不清,因为我们压根就不知道真实的宇宙模型到底是什么。 这个与问题真实解之间的误差,就叫做风险(更严格的说,误差的累积叫做风险)。我们选择了一个假设之后(更直观点说,我们得到了一个分类器以后),真实误差无从得知,但我们可以用某些可以掌握的量来逼近它。最直观的想法就是使用分类器在样本数据上的分类的结果与真实结果(因为样本是已经标注过的数据,是准确的数据)之间的差值来表示。这个差值叫做经验风险Remp(w)。以前的机器学习方法都把经验风险最小化作为努力的目标,但后来发现很多分类函数能够在样本集上轻易达到100%的正确率,在真实分类时却一塌糊涂(即所谓的推广能力差,或泛化能力差)。此时的情况便是选择了一个足够复杂的分类函数(它的VC维很高),能够精确的记住每一个样本,但对样本之外的数据一律分类错误。回头看看经验风险最小化原则我们就会发现,此原则适用的大前提是经验风险要确实能够逼近真实风险才行(行话叫一致),但实际上能逼近么?答案是不能,因为样本数相对于现实世界要分类的文本数来说简直九牛

20.ENVI4.3 支持向量机分类原理、操作及实例分析

ENVI4.3 支持向量机分类原理、操作及实例分析 一、支持向量机算法介绍 1.支持向量机算法的理论背景 支持向量机分类(Support Vector Machine或SVM)是一种建立在统计学习理论(Statistical Learning Theory或SLT)基础上的机器学习方法。 与传统统计学相比,统计学习理论(SLT)是一种专门研究小样本情况下及其学习规律的理论。该理论是建立在一套较坚实的理论基础之上的,为解决有限样本学习问题提供了一个统一的框架。它能将许多现有方法纳入其中,有望帮助解决许多原来难以解决的问题,如神经网络结构选择问题、局部极小点问题等;同时,在这一理论基础上发展了一种新的通用学习方法——支持向量机(SVM),已初步表现出很多优于已有方法的性能。一些学者认为,SLT和SVM正在成为继神经网络研究之后新的研究热点,并将推动机器学习理论和技术的重大发展。 支持向量机方法是建立在统计学习理论的VC维(VC Dimension)理论和结构风险最小原理基础上的,根据有限的样本信息在模型的复杂性(即对特定训练样本的学习精度)和学习能力(即无错误地识别任意样本的能力)之间寻求最佳折衷,以期获得最好的推广能力。 支持向量机的几个主要优点有: (1)它是专门针对有限样本情况的,其目标是得到现有信息下的最优解而不仅仅是样本数趋于无穷大时的最优值; (2)算法最终将转化成为一个二次型寻优问题,从理论上说,得到的将是全局最优点,解决了在神经网络方法中无法避免的局部极值问题; (3)算法将实际问题通过非线性变换转换到高维的特征空间(Feature Space),在高维空间中构造线性判别函数来实现原空间中的非线性判别函数,特殊性质能保证机器有较 好的推广能力,同时它巧妙地解决了维数问题,其算法复杂度与样本维数无关; 2.支持向量机算法简介 通过学习算法,SVM可以自动寻找那些对分类有较大区分能力的支持向量,由此构造出分类器,可以将类与类之间的间隔最大化,因而有较好的推广性和较高的分类准确率。 最优分类面(超平面)和支持向量

支持向量机(SVM)原理及应用概述

东北大学 研究生考试试卷 考试科目:信号处理的统计分析方法 课程编号: 09601513 阅卷人: 刘晓志 考试日期: 2012年11月07日 姓名:赵亚楠 学号: 1001236 注意事项 1.考前研究生将上述项目填写清楚.

2.字迹要清楚,保持卷面清洁. 3.交卷时请将本试卷和题签一起上交. 4.课程考试后二周内授课教师完成评卷工作,公共课成绩单与试卷交 研究生院培养办公室,专业课成绩单与试卷交各学院,各学院把成 绩单交研究生院培养办公室. 东北大学研究生院培养办公室 支持向量机(SVM)原理及应用 目录 一、SVM的产生与发展 (3) 二、支持向量机相关理论 (4) (一)统计学习理论基础 (4) (二)SVM原理 (4) 1.最优分类面和广义最优分类面 (5) 2.SVM的非线性映射 (7)

3.核函数 (8) 三、支持向量机的应用研究现状 (9) (一)人脸检测、验证和识别 (10) (二)说话人/语音识别 (10) (三)文字/手写体识别 (11) (四)图像处理 (11) (五)其他应用研究 (12) 四、结论和讨论 (12) 支持向量机(SVM )原理及应用 一、SVM 的产生与发展 自1995年Vapnik 在统计学习理论的基础上提出SVM 作为模式识别的新方法之后,SVM 一直倍受关注。同年,Vapnik 和Cortes 提出软间隔(soft margin)SVM ,通过引进松弛变量i ξ度量数据i x 的误分类(分类出现错误时i ξ大于0),同时在目 标函数中增加一个分量用来惩罚非零松弛变量(即代价函数),SVM 的寻优过程即

支持向量机(SVM)原理及

支持向量机(SVM)原理及应用概述

支持向量机(SVM )原理及应用 一、SVM 的产生与发展 自1995年Vapnik (瓦普尼克)在统计学习理论的基础上提出SVM 作为模式识别的新方法之后,SVM 一直倍受关注。同年,Vapnik 和Cortes 提出软间隔(soft margin)SVM ,通过引进松弛变量i ξ度量数据i x 的误分类(分类出现错误时i ξ大于0),同时在目标函数中增加一个分量用来惩罚非零松弛变量(即代价函数),SVM 的寻优过程即是大的分隔间距和小的误差补偿之间的平衡过程;1996年,Vapnik 等人又提出支持向量回归 (Support Vector Regression ,SVR)的方法用于解决拟合问题。SVR 同SVM 的出发点都是寻找最优超平面(注:一维空间为点;二维空间为线;三维空间为面;高维空间为超平面。),但SVR 的目的不是找到两种数据的分割平面,而是找到能准确预测数据分布的平面,两者最终都转换为最优化问题的求解;1998年,Weston 等人根据SVM 原理提出了用于解决多类分类的SVM 方法(Multi-Class Support Vector Machines ,Multi-SVM),通过将多类分类转化成二类分类,将SVM 应用于多分类问题的判断:此外,在SVM 算法的基本框架下,研究者针对不同的方面提出了很多相关的改进算法。例如,Suykens 提出的最小二乘支持向量机 (Least Square Support Vector Machine ,LS —SVM)算法,Joachims 等人提出的SVM-1ight ,张学工提出的中心支持向量机 (Central Support Vector Machine ,CSVM),Scholkoph 和Smola 基于二次规划提出的v-SVM 等。此后,台湾大学林智仁(Lin Chih-Jen)教授等对SVM 的典型应用进行总结,并设计开发出较为完善的SVM 工具包,也就是LIBSVM(A Library for Support Vector Machines)。LIBSVM 是一个通用的SVM 软件包,可以解决分类、回归以及分布估计等问题。 二、支持向量机原理 SVM 方法是20世纪90年代初Vapnik 等人根据统计学习理论提出的一种新的机器学习方 法,它以结构风险最小化原则为理论基础,通过适当地选择函数子集及该子集中的判别函数,使学习机器的实际风险达到最小,保证了通过有限训练样本得到的小误差分类器,对独立测试集的测试误差仍然较小。 支持向量机的基本思想:首先,在线性可分情况下,在原空间寻找两类样本的最优分类超平面。在线性不可分的情况下,加入了松弛变量进行分析,通过使用非线性映射将低维输

随机森林与支持向量机分类性能比较

随机森林与支持向量机分类性能比较 黄衍,查伟雄 (华东交通大学交通运输与经济研究所,南昌 330013) 摘要:随机森林是一种性能优越的分类器。为了使国内学者更深入地了解其性能,通过将其与已在国内得到广泛应用的支持向量机进行数据实验比较,客观地展示其分类性能。实验选取了20个UCI数据集,从泛化能力、噪声鲁棒性和不平衡分类三个主要方面进行,得到的结论可为研究者选择和使用分类器提供有价值的参考。 关键词:随机森林;支持向量机;分类 中图分类号:O235 文献标识码: A Comparison on Classification Performance between Random Forests and Support Vector Machine HUANG Yan, ZHA Weixiong (Institute of Transportation and Economics, East China Jiaotong University, Nanchang 330013, China)【Abstract】Random Forests is an excellent classifier. In order to make Chinese scholars fully understand its performance, this paper compared it with Support Vector Machine widely used in China by means of data experiments to objectively show its classification performance. The experiments, using 20 UCI data sets, were carried out from three main aspects: generalization, noise robustness and imbalanced data classification. Experimental results can provide references for classifiers’ choice and use. 【Key words】Random Forests; Support Vector Machine; classification 0 引言 分类是数据挖掘领域研究的主要问题之一,分类器作为解决问题的工具一直是研究的热点。常用的分类器有决策树、逻辑回归、贝叶斯、神经网络等,这些分类器都有各自的性能特点。本文研究的随机森林[1](Random Forests,RF)是由Breiman提出的一种基于CART 决策树的组合分类器。其优越的性能使其在国外的生物、医学、经济、管理等众多领域到了广泛的应用,而国内对其的研究和应用还比较少[2]。为了使国内学者对该方法有一个更深入的了解,本文将其与分类性能优越的支持向量机[3](Support Vector Machine,SVM)进行数据实验比较,客观地展示其分类性能。本文选取了UCI机器学习数据库[4]的20个数据集作为实验数据,通过大量的数据实验,从泛化能力、噪声鲁棒性和不平衡分类三个主要方面进行比较,为研究者选择和使用分类器提供有价值的参考。 1 分类器介绍 1.1 随机森林 随机森林作为一种组合分类器,其算法由以下三步实现: 1. 采用bootstrap抽样技术从原始数据集中抽取n tree个训练集,每个训练集的大小约为原始数据集的三分之二。 2. 为每一个bootstrap训练集分别建立分类回归树(Classification and Regression Tree,CART),共产生n tree棵决策树构成一片“森林”,这些决策树均不进行剪枝(unpruned)。在作者简介:黄衍(1986-),男,硕士研究生,主要研究方向:数据挖掘与统计分析。 通信联系人:查伟雄,男,博士,教授,主要研究方向:交通运输与经济统计分析。 E-mail: huangyan189@https://www.sodocs.net/doc/3e4030327.html,.

支持向量机理论及工程应用实例

《支持向量机理论及工程应用实例》 支持向量机理论及工程应用实例 求助编辑百科名片 《支持向量机理论及工程应用实例》共分为8章,从机器学习的基本问题开始,循序渐进地介绍了相关的内容,包括线性分类器、核函数特征空间、推广性理论和优化理论,从而引出了支持向量机的算法,进而将支持向量机应用到实际的工程实例中。《支持向量机理论及工程应用实例》适合高等院校高年级本科生、研究生、教师和相关科研人员及相关领域的工作者使用。《支持向量机理论及工程应用实例》既可作为研究生教材,也可作为神经网络、机器学习、数据挖掘等课程的参考教材。 书名: 支持向量机理论及工程应用实例 作者: 白鹏 张斌 ISBN : 9787560620510 定价: 16.00 元 出版社: 西安电子科技大学出版社 出版时间: 2008 开本: 16 LIBSVM 的简单介绍 2006-09-20 15:59:48 大 中 小 1. LIBSVM 软件包简介 LIBSVM 是台湾大学林智仁(Chih-Jen Lin)博士等开发设计的一个操作简单、易于使用、快速有效的通用SVM 软件包,可以解决分类问题(包括C- SVC 、n - SVC )、回归问题(包括e - SVR 、n - SVR )以及分布估计 (one-class-SVM )等问题,提供了线性、多项式、径向基和S 形函数四种常

用的核函数供选择,可以有效地解决多类问题、交叉验证选择参数、对不平衡样本加权、多类问题的概率估计等。LIBSVM 是一个开源的软件包,需要者都可以免费的从作者的个人主页 处获得。他不仅提供了LIBSVM的C++语言的算法源代码,还提供了Python、Java、R、MATLAB、Perl、Ruby、LabVIEW以及C#.net 等各种语言的接口,可以方便的在Windows 或UNIX 平台下使用。另外还提供了WINDOWS 平台下的可视化操作工具SVM-toy,并且在进行模型参数选择时可以绘制出交叉验证精度的等高线图。 2. LIBSVM 使用方法简介 LibSVM是以源代码和可执行文件两种方式给出的。如果是Windows系列操作系统,可以直接使用软件包提供的程序,也可以进行修改编译;如果是Unix类系统,必须自己编译。 LIBSVM 在给出源代码的同时还提供了Windows操作系统下的可执行文件,包括:进行支持向量机训练的svmtrain.exe;根据已获得的支持向量机模型对数据集进行预测的svmpredict.exe;以及对训练数据与测试数据进行简单缩放操作的svmscale.exe。它们都可以直接在DOS 环境中使用。如果下载的包中只有C++的源代码,则也可以自己在VC等软件上编译生成可执行文件。 3. LIBSVM 使用的一般步骤是: 1)按照LIBSVM软件包所要求的格式准备数据集; 2)对数据进行简单的缩放操作; 3)考虑选用RBF 核函数; 4)采用交叉验证选择最佳参数C与g ; 5)采用最佳参数C与g 对整个训练集进行训练获取支持向量机模型; 6)利用获取的模型进行测试与预测。 4. LIBSVM使用的数据格式 1)训练数据和检验数据文件格式如下:

相关主题