numpy - Matrix completion in Python -


say have matrix:

> import numpy nap > = np.random.random((5,5))  array([[ 0.28164485,  0.76200749,  0.59324211,  0.15201506,  0.74084168],        [ 0.83572213,  0.63735993,  0.28039542,  0.19191284,  0.48419414],        [ 0.99967476,  0.8029097 ,  0.53140614,  0.24026153,  0.94805153],        [ 0.92478   ,  0.43488547,  0.76320656,  0.39969956,  0.46490674],        [ 0.83315135,  0.94781119,  0.80455425,  0.46291229,  0.70498372]])

and punch holes in np.nan, e.g.:

> a[(1,4,0,3),(2,4,2,0)] = np.nan;   array([[ 0.80327707,  0.87722234,         nan,  0.94463778,  0.78089194],        [ 0.90584284,  0.18348667,         nan,  0.82401826,  0.42947815],        [ 0.05913957,  0.15512961,  0.08328608,  0.97636309,  0.84573433],        [        nan,  0.30120861,  0.46829231,  0.52358888,  0.89510461],        [ 0.19877877,  0.99423591,  0.17236892,  0.88059185,        nan ]])

i fill-in nan entries using information rest of entries of matrix. example using average value of column nan entries occur.

more generally, there libraries in python matrix completion ? (e.g. along lines of candes & recht's convex optimization method).

background:

this problem appears in machine learning. example when working missing features in classification/regression or in collaborative filtering (e.g. see netflix problem on wikipedia , here)

if install latest scikit-learn, version 0.14a1, can use shiny new imputer class:

>>> sklearn.preprocessing import imputer >>> imp = imputer(strategy="mean") >>> = np.random.random((5,5)) >>> a[(1,4,0,3),(2,4,2,0)] = np.nan >>> array([[ 0.77473361,  0.62987193,         nan,  0.11367791,  0.17633671],        [ 0.68555944,  0.54680378,         nan,  0.64186838,  0.15563309],        [ 0.37784422,  0.59678177,  0.08103329,  0.60760487,  0.65288022],        [        nan,  0.54097945,  0.30680838,  0.82303869,  0.22784574],        [ 0.21223024,  0.06426663,  0.34254093,  0.22115931,         nan]]) >>> = imp.fit_transform(a) >>> array([[ 0.77473361,  0.62987193,  0.24346087,  0.11367791,  0.17633671],        [ 0.68555944,  0.54680378,  0.24346087,  0.64186838,  0.15563309],        [ 0.37784422,  0.59678177,  0.08103329,  0.60760487,  0.65288022],        [ 0.51259188,  0.54097945,  0.30680838,  0.82303869,  0.22784574],        [ 0.21223024,  0.06426663,  0.34254093,  0.22115931,  0.30317394]])

after this, can use imp.transform same transformation other data, using mean imp learned a. imputers tie scikit-learn pipeline objects can use them in classification or regression pipelines.

if want wait stable release, 0.14 should out next week.

full disclosure: i'm scikit-learn core developer


Comments

Post a Comment

Popular posts from this blog

basic authentication with http post params android -

i doing basic authentication passing username , password , using basicnamevaluepair sending post params response service. my method: public stringbuilder callservicehttppost(string username, string password, string type) { // create new httpclient , post header httpclient httpclient = new defaulthttpclient(); httppost httppost = new httppost(webservice + type); httpresponse response = null; stringbuilder total = new stringbuilder(); try { url url = new url(webservice + type); /*string base64encodedcredentials = base64.encodetostring((username + ":" + password).getbytes(), base64.url_safe | base64.no_wrap);*/ string base64encodedcredentials = "basic " + base64.encodetostring( (username + ":" + password).getbytes(), base64.no_wrap); httppost.setheader("auth...
Read more

vb.net - Virtual Keyboard commands -

i wondering difference between virtual these keyboard commands is: keyeventf_extendedkey , keyeventf_keyup is. everywhere have looked gives me description based off integers , not want know each of them does. you've tagged question vb.net, these have nothing @ vb.net. they're constants defined in windows header files, use win32 api functions. as far difference, can't tell looking @ values. individual values not particularly important, that's why named identifiers used. what's important used , documentation functions tells mean. the first one, keyeventf_extendedkey , used keybdinput structure (which used along e.g. sendinput function) pass information synthesized keyboard input. if flag used, means scan code should interpreted extended key. technically, means scan code preceded prefix byte value 224 (&he0 in hexadecimal notation). the second one, keyeventf_keyup , 1 of flags available use structure. means key being released (going up)...
Read more

How to get multiresult with multicondition in Sql Server -

i need solve this. hopefully can giving me advices. for sample, i've got data : proclib.march 1 first 10 rows flight date depart orig dest miles boarded capacity ----------------------------------------------------------------- 114 01mar94 7:10 lga lax 2475 172 210 202 01mar94 10:43 lga ord 740 151 210 219 01mar94 9:31 lga lon 3442 198 250 622 01mar94 12:19 lga fra 3857 207 250 132 01mar94 15:35 lga yyz 366 115 178 271 01mar94 13:17 lga par 3635 138 250 302 01mar94 20:22 lga 229 105 180 114 02mar94 7:10 lga lax 2475 119 210 202 02mar94 10:43 lga ord 740 120 210 219 02mar94 9:31 lga lon 344...
Read more