Data frames containing factor, character or logical columns can be passed
directly to midas_fit() and midas(). Previously any such
frame was rejected by the server with an HTTP 422 "float parsing" error,
because the client serialised data through as.matrix(), which coerces a
mixed data frame to a character matrix.
Categorical data are now sent using the Parquet transport, which preserves column types, so the MIDAS engine fits a softmax head over the levels of a multi-level factor and a binary head over a two-level one -- rather than requiring the user to one-hot encode and thereby modelling the resulting dummies as independent Bernoulli variables.
arrow package (in Suggests). If arrow is not
installed, categorical data now raise a clear, actionable error naming the
package to install, rather than an opaque HTTP error from the server.midas_transform() and midas() now return data frames whose columns have
the classes passed to midas_fit(), including factor levels and ordering.
Previously every returned column was character, because the response was
rebuilt with do.call(rbind, ...), which coerces to a character matrix as
soon as one column is non-numeric.
The imputation engine ingests data as float32 and returns observed cells from that downcast copy, so an observed numeric value came back as the binary32 round-trip of the input -- a relative change of about 6e-8. That is invisible on standardised covariates but amounts to roughly 1e-4 on data of order 1000. Nothing was being re-estimated; the loss was purely the representation change on ingest.
midas_transform() gains a data argument. Pass the frame given to
midas_fit() and every cell that was observed in it is restored to its
original value, so only missing cells differ from the input.midas() does this automatically, since it is handed the data.combine() returned every column of the Rubin's rules table -- including
estimate, std.error, statistic, df and p.value -- as character.
These are now numeric, so the result can be used directly without manual
coercion. This affected all models, not only those with categorical data.imp_mean() now fails with an explanatory message when the model contains
categorical columns, instead of surfacing an HTTP 500 from the server. An
element-wise mean over factor levels is not defined; use
midas_transform() and pool in the manner appropriate to the analysis.to_nested_list() now validates its input and reports a clear internal
error rather than silently coercing non-numeric columns..Rbuildignore now excludes .Rhistory files in subdirectories, not only
at the package root.?midas_fit gains a "Column types" section describing how each R class is
modelled, and a "Choosing dropout_prob" section. The default of 0.5 suits
wide data; on narrow frames (fewer than roughly 15 columns) a smaller value
such as 0.2 can substantially improve reconstruction accuracy. The default
is unchanged.start_server(): Launch the MIDAS2 Python API server.stop_server(): Stop the running server.ensure_server(): Start the server if it is not already running.midas_fit(): Fit a MIDAS denoising autoencoder model.midas_transform(): Generate multiple imputations from a fitted model.midas(): All-in-one fit + transform convenience function.imp_mean(): Compute element-wise mean across imputations.combine(): Run Rubin's rules regression across imputations.overimpute(): Overimputation diagnostic (mask-and-reimpute RMSE).install_backend(): Install the Python backend into an isolated environment.