Vol: 14, Issue: 1 2024

Page: 49-58

Combining AI Paradigms for Effective Data Imputation: A Hybrid Approach

Arunkumar Thirunagalingam

Received Date: 2023-12-17

Accepted Date: 2024-01-18

Published Date: 2024-03-11

http://doi.org/10.37648/ijtbm.v14i01.007

In data analysis, data imputation is an essential procedure, especially when working with partial datasets. Machine learning models' validity and performance can be significantly impacted by missing data. Conventional techniques for data imputation, including regression models or mean/mode imputation, frequently fall short of capturing the complex relationships present in the data. In order to increase the precision and resilience of data imputation, this research suggests a hybrid methodology that integrates several AI paradigms, such as machine learning, deep learning, and statistical techniques. The suggested hybrid strategy performs better than traditional methods in a variety of contexts, according to experimental results, providing a more dependable way to handle missing data in complicated datasets.

References

• A. Gelman and J. Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge, U.K.: Cambridge Univ. Press, 2007
• G. H. Chen and D. Zhou, "A Study on Missing Data Imputation," in Proc. 17th Int. Conf. Artif. Intell. Statist., 2014, pp. 87-94.
• Y. Bengio, A. Courville, and P. Vincent, "Representation Learning: A Review and New Perspectives," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1798-1828, Aug. 2013.
• . J. Schmidhuber, "Deep Learning in Neural Networks: An Overview," Neural Networks, vol. 61, pp. 85-117, Jan. 2015.
• M. T. Ribeiro, S. Singh, and C. Guestrin, "Why Should I Trust You?": Explaining the Predictions of Any Classifier," in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2016, pp. 1135-1144.
• . D. P. Kingma and M. Welling, "Auto-Encoding Variational Bayes," in Proc. 2nd Int. Conf. Learn. Representations, 2014.
• S. Thrun and L. Pratt, Eds., Learning to Learn. Boston, MA: Springer, 1998.
• G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions. Hoboken, NJ: Wiley, 2008.
• T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. New York, NY: Springer, 2009.
• Z. Ghahramani and M. I. Jordan, "Supervised Learning from Incomplete Data via an EM Approach," in Advances in Neural Information Processing Systems (NIPS), vol. 6, Denver, CO, 1994, pp. 120-127.
• . J. Little and D. B. Rubin, Statistical Analysis with Missing Data, 2nd ed. Hoboken, NJ: Wiley, 2002.
• S. Buuren and K. Groothuis-Oudshoorn, "mice: Multivariate Imputation by Chained Equations in R," J. Stat. Softw., vol. 45, no. 3, pp. 1-67, Dec. 2011.
• Y. Rubinstein and T. Hastie, "Discriminative vs Informative Learning," in Proc. 22nd Int. Conf. Machine Learning (ICML), 2005, pp. 209-216.
• S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735- 1780, Nov. 1997.
• A. Gelman, X. L. Meng, and H. Stern, "Posterior Predictive Assessment of Model Fitness via Realized Discrepancies," Statistica Sinica, vol. 6, no. 4, pp. 733-807, Oct. 1996.
• I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA: MIT Press, 2016.
• B. Lakshminarayanan, A. Pritzel, and C. Blundell, "Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles," in Proc. 31st Conf. Neural Inf. Process. Syst. (NIPS), 2017, pp. 6402-6413.
• . D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
• . M. Abadi et al., "TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems," in Proc. 12th USENIX Symp. Oper. Syst. Design Implement. (OSDI), 2016, pp. 265-283.
• K. K. Singh and Y. Upadhyay, "Handling Missing Data in Machine Learning: A Review," in Proc. Int. Conf. Computational Intelligence and Data Science (ICCIDS), 2020, pp. 180-185.