numpy - plotting a 2D matrix in python, code and most useful visualization -

-

i have large matrix(10x55678) in "numpy" matrix format. rows of matrix correspond "topics" , columns correspond words(unique words text corpus). each entry i,j in matrix probability, meaning word j belongs topic probability x. since using ids rather real words , since dimension of matrix large need visualized in way.which visualization suggest? simple plot? or more sophisticated , informative one?(i asking these cause ignorant useful types of visualization). if possible can give me example using numpy matrix? thanks

the reason asked question want have general view of word-topic distributions in corpus. other methods welcome

you use matplotlib's imshowor pcolor method display data, comments have mentioned, might hard interpret without zooming in on subsets of data.

a = np.random.normal(0.0,0.5,size=(5000,10))**2 = a/np.sum(a,axis=1)[:,none]  # normalize  pcolor(a)

unsorted random example

you sort words probability belong cluster:

maxvi = np.argsort(a,axis=1) ii = np.argsort(maxvi[:,-1])  pcolor(a[ii,:])

enter image description here

here word index on y-axis no longer equals original ordering since things have been sorted.

another possibility use networkx package plot word clusters each category, words highest probability represented nodes either larger or closer center of graph , ignore words have no membership in category. might easier since have large number of words , small number of categories.

hopefully 1 of these suggestions useful.


Comments

Post a Comment

Popular posts from this blog

c# - Is it possible to remove an existing registration from Autofac container builder? -

-
something along lines: builder.registertype<mytype>().as<itype>(); builder.registertype<mytype2>().as<itype>(); builder.deregistertype<mytype>().as<itype>() var container = builder.build(); var types = container.resolve<ienumerable<itype>>(); assert.istrue(types.count == 1); assert.istrue(types[0].gettype == typeof(mytype2)); scenario: go through bunch of assemblies , go register types want make sure have 1 implementation of given type. need before create container. track on own nice if autofac me bit. this cannot done directly using containerbuilder , unless start on new one. mind you, having first built container should able construct new container filtering away unwanted types , reusing registrations first container. this: ... var container = builder.build(); builder = new containerbuilder(); var components = container.componentregistry.registrations .where(cr => cr.activator.limittype != typ...
Read more

php - Mysql PK and FK char(36) vs int(10) -

-
i have read mysql can handle indexes , searches better if tables keys integers rather char(36) uuid's. is worth converting int(10)? note: use 1 server , have have plans use multiple databases concurrently therefore removing 1 of big reasons using uuid. our tables innodb if makes difference. thanks in advance. usually rdbms can handle integer keys pk more efficient, other datatypes. reason how build index column, yes: long dont require string (or other typed) keys, should use integers. however: char(36) , int(10) far away being equal, because int(10) much smaller, char(36). dont know, if require many different keys, should keep in mind. update, complete last paragraph: int(10) 32-bit, char(36) 36 - ^byte^ (= 288-bit). not mean, int consumes less space, means, char(36) has 4 times more different keys.
Read more