Suppose I have this data frame: times vals 1 1 2 2 3 4 3 7 6 set up with foo <- data.frame(times=c(1,3,7), vals=c(2,4,6)) and I want this one: times vals 1 1 2 2 2 2 3 3 4 4 4 4 5 5 4 6 6 4 7 7 6 That is, I … Read more
I have a Pandas dataframe where I have designated some of the columns as indices: planets_dataframe.set_index([‘host’,’name’], inplace=True) and would like to be able to refer to these indices in a variety of contexts. Using the name of an index works fine in queries planets_dataframe.query(‘host == “PSR 1257 12″‘) but results in an error if try … Read more
So I have a data frame that has a date column, an hour column and a series of other numerical columns. Each row in the data frame is 1 hour of 1 day for an entire year. The data frame looks like this: Date Hour Melbourne Southern Flagstaff 1 2009-05-01 0 0 5 17 2 … Read more
I have a recurrent situation where I set a value at the top of a long set of R code that’s used in subsetting one or more data frames. Something like this: city_code <- “202” At the end of the whole process I’d like to save the results in a data frame that’s named appropriately, … Read more
I have multiple pandas data frame objects cost1, cost2, cost3 …. They have different column names (and number of columns) but have some in common. Number of columns is fairly large in each data frame, hence handpicking the common columns manually will be painful. How can I append rows from all of these data frames … Read more
The Spark documentation shows how to create a DataFrame from an RDD, using Scala case classes to infer a schema. I am trying to reproduce this concept using sqlContext.createDataFrame(RDD, CaseClass), but my DataFrame ends up empty. Here’s my Scala code: // sc is the SparkContext, while sqlContext is the SQLContext. // Define the case class … Read more
When trying to count rows with similar ‘kind’ in data frame: import pandas as pd items = [(‘aaa’,’aaa text 1′), (‘aaa’,’aaa text 2′), (‘aaa’,’aaa text 3′), (‘bb’, ‘bb text 1’), (‘bb’, ‘bb text 2’), (‘bb’, ‘bb text 3’), (‘bb’, ‘bb text 4’), (‘cccc’,’cccc text 1′), (‘cccc’,’cccc text 2′), (‘dd’, ‘dd text 1’), (‘e’, ‘e text … Read more
This question already has answers here: How to create a dictionary of two pandas DataFrame columns (5 answers) Convert a Pandas DataFrame to a dictionary (11 answers) Closed 3 years ago. I have the dataframe ID A B C 0 p 1 3 2 1 q 4 3 2 2 r 4 0 9 And … Read more
Is there any way to join a Series to a DataFrame directly? The join would be on a field of the dataframe and on the index of the series. The only way I found was to convert the series to a dataframe first, as in the code below. import numpy as np import pandas as … Read more
Many R users eventually figure out lots of ways to remove elements from their data. One way is to use NULL, particularly when you want to do something like drop a column from a data.frame or drop an element from a list. Eventually, a user comes across a situation where they want to drop several … Read more
