In this paper, we investigate whether graph kernels can be used as a means of analyzing time-varying financial market networks. Specifically, we aim to identify the significant financial incident that changes the financial network properties through graph kernels. Our financial networks are abstracted from the New York Stock Exchange (NYSE) data over 6004 trading days, where each vertex represents the individual daily return price time series of a stock and each edge represents the correlation between pairwise series. We propose to use two state-of-the-art graph kernels for the analysis, i.e., the Jensen-Shannon graph kernel and the Weisfeiler-Lehman subtree kernel. The reason of using the two kernels is that they are the representative methods of global graph kernels and local graph kernels, respectively. We perform kernel Principle Components Analysis (kPCA) associated with each kernel matrix to embed the networks into a 3-dimensional principle space, where the time-varying networks of all trading days are visualized. Experimental results on the financial time series of NYSE dataset demonstrate that graph kernels can well distinguish abrupt changes of financial networks with time, and provide a more effective alternative way of analyzing original multiple co-evolving financial time series. We theoretically indicate the perspective of developing novel graph kernels on time-varying networks for multiple co-evolving time series analysis in future work.