Corpus Linguistics and Naive Discriminative Learning
Keywords:
machine learning, dative alternation, Switchboard, probabilistic distributional patterns, collexeme analysisAbstract
Three classifiers from machine learning (the generalized linear mixed model, memory based learning, and support vector machines) are compared with a naive discriminative learning classifier, derived from basic principles of error-driven learning characterizing animal and human learning. Tested on the dative alternation in English, using the Switchboard data from (BRESNAN; CUENI; NIKITINA; BAAYEN, 2007), naive discriminative learning emerges with state-of-the-art predictive accuracy. Naive discriminative learning offers a united framework for understanding the learning of probabilistic distributional patterns, for classification, and for a cognitive grounding of distinctive collexeme analysis.
Downloads
References
ALBRIGHT, A.; HAYES, B. Rules vs. analogy in English past tenses: A computational/experimental study. Cognition, v. 90, p. 119-161, 2003.
ALLAN, L. G. A note on measurement of contingency between two binary variables in judgment tasks. Bulletin of the Psychonomic Society, v. 15, p. 147-149, 1980.
ANDERSON, J. R. Learning and memory: An integrated approach. New York: Wiley, 2000.
ARNON, I.; SNIDER, N. Syntactic probabilities affect pronunciation variation in spontaneous speech. Journal of Memory and Language, v. 62, p. 67-82, 2010.
ARPPE, A.; BAAYEN, R. H. ndl: Naive discriminative learning: an implementation in r [Computer software manual]. 2011. Available at: <http://CRAN.R-project.org/package=ndl (R package version 0.4)>
BAAYEN, R. H. languageR: Data sets and functions with "analyzing linguistic data:A practical introduction to statistics". [Computer software manual]. 2009. Available at: <http://CRAN.R-project.org/package=languageR (R package version 0.955)>
BAAYEN, R. H.; HENDRIX, P. Sidestepping the combinatorial explosion: Towards a processing model based on discriminative learning. Empirically examining parsimony and redundancy in usage-based models, LSA workshop, January 2011.
BAAYEN, R. H.; MILIN, P.; FILIPOVIC DURDJEVIC, D.; HENDRIX, P.; MARELLI, M. An amorphous model for morphological processing in visual comprehension based on naive discriminative learning. Psychological Review, 2011. (in press).
BATES, D.; MAECHLER, M. lme4: Linear mixed-effects models using s4 classes [Computer software manual]. 2009. Available at: <http://CRAN.Rproject.org/package=lme4 (R package version 0.999375-31)>
BATES, E.; MacWHINNEY, B. Competition, variation, and language learning. In: MacWHINNEY, B. (Ed.). Mechanisms of language acquisition. Hillsdale, NJ: Lawrence Erlbalm Assoc., 1987.
BOD, R. Exemplar-based syntax: How to get productivity from examples. The Linguistic Review, v. 23, n. 3, p. 291-320, 2006.
BOERSMA, P.; HAYES, B. Empirical tests of the gradual learning algorithm. Linguistic Inquiry, v. 32, p. 45-86, 2001.
BREIMAN, L. Random forests. Machine Learning, v. 45, p. 5-32, 2001.
BRESNAN, J.; CUENI, A.; NIKITINA, T.; BAAYEN, R. H. Predicting the dative alternation. In: Bouma, G.; KRAEMER, I.; ZWARTS, J. (Ed.). Cognitive foundations of interpretation Royal Netherlands Academy of Arts and Sciences, 2007.
CEDERGREN, H.; SANKOFF, D. Variable rules: Performance as a statistical reflection of competence. Language, v. 50, n. 2, p. 333-355, 1974.
CHATER, N.; TENENBAUM, J. B.; YUILLE, A. Probabilistic models of cognition: Conceptual foundations. Trends in Cognitive Science, v. 10, n. 7, p. 287-291, 2006.
COLTHEART, M.; RASTLE, K.; PERRY, C.; LANGDON, R.; ZIEGLER, J. DRC: A dual route cascaded model of visual word recognition and reading aloud. Psychological Review, v. 108, n. 1, p. 204-256, 2001.
DAELEMANS, W.; BOSCH, A. Van den. Memory-based language processing Cambridge: Cambridge University Press, 2005.
DAELEMANS, W.; ZAVREL, J.; SLOOT, K. Van der; BOSCH, A. Van den. TiMBL: Tilburg Memory Based Learner Reference Guide. Version 6.3 (Technical Report No. ILK 10-01). 2010. Computational Linguistics Tilburg University.
DANKS, D. Equilibria of the Rescorla-Wagner model. Journal of Mathematical Psychology, v. 47, n. 2, p. 109-121, 2003.
DAW, N.; SHOHAMY, D. The cognitive neuroscience of motivation and learning. Social Cognition, v. 26, n. 5, p. 593-620, 2008.
DAYAN, P.; KAKADE, S. Explaining away in weight space. In: LEEN, T. K.; DIETTERICH, T. G.; TRESP, V. (Ed.). Advances in neural information processingsystems 13 Cambridge, MA: MIT Press, 2001.
DIMITRIADOU, E.; HORNIK, K.; LEISCH, F.; MEYER, D.; WEINGESSEL, A. e1071: Misc Functions of the Department of Statistics (e1071), TU Wien [Computer software manual]. 2009. (R package version 1.5-19)
ELLIS, N. C. Language acquisition as rational contingency learning. AppliedLinguistics, v. 27, n. 1, p. 1-24, 2006.
FORD, M.; BRESNAN, J. Predicting syntax: Processing dative constructions in American and Australian varieties of English. Language, v. 86, n. 1, p. 168-213, 2010.
GLUCK, M. A.; BOWER, G. H. From conditioning to category learning: An adaptive network model. Journal of Experimental Psychology: General, v. 117, n. 3, p. 227-247, 1988.
GRIES, St. Th. Frequency tables: tests, effect sizes, and explorations. In: GLYNN, D.; ROBINSON, J. (Ed.). Polisemy and synonymy: Corpus methods and applications in Cognitive Linguistics. Amsterdam / Philadelphia: John Benjamins, 2011.
GRIES, St. Th.; STEFANOWITSCH, A. Extending collostructional analysis: A corpus-based perspective on alternations. International Journal of CorpusLinguistics, v. 9, n. 1, p. 97-129, 2004.
HARM, M. W.; SEIDENBERG, M. S. Computing the meanings of words in reading: Cooperative division of labor between visual and phonological processes. Psychological Review, v. 111, p. 662-720, 2004.
HARRELL, F. Regression modeling strategies Berlin: Springer, 2001.
HSU, A. S.; CHATER, N.; VITÁNYI, P. The probabilistic analysis of language acquisition: Theoretical, computational, and experimental analysis, 2010. Manuscript submitted for publication.
LEVELT, W. J. M.; ROELOFS, A.; MEYER, A. S. A theory of lexical access in speech production. Behavioral and Brain Sciences, v. 22, p. 1-38, 1999.
MacWHINNEY, B. A united model of language acquisition. In: KROLL, J.; GROOT, A. de (Ed.). Handbook of bilingualism: Psycholinguistic approaches Oxford University Press, 2005.
McCLELLAND, J. L.; RUMELHART, D. E. An interactive activation model of context effects in letter perception: Part I. An account of the basic findings. Psychological Review, v. 88, p. 375-407, 1981.
MILLER, R. R.; BARNET, R. C.; GRAHAME, N. J. Assessment of the Rescorla-Wagner Model. Psychological Bulletin, v. 117, n. 3, p. 363-386, 1995.
MURRAY, W. S.; FORSTER, K. Serial mechanisms in lexical access: the rank hypothesis. Psychological Review, v. 111, p. 721-756, 2004.
NORRIS, D. The Bayesian Reader: Explaining Word Recognition as an Optimal Bayesian Decision Process. Psychological Review, v. 113, n. 2, p. 327-357, 2006.
NORRIS, D.; McQUEEN, J. Shortlist B: A Bayesian model of continuous speech recognition. Psychological Review, v. 115, n. 2, p. 357-395, 2008.
RAMSCAR, M.; DYE, M.; POPICK, H. M.; O'DONNELL-McCARTHY, F. The Right Words or Les Mots Justes? Why Changing the Way We Speak to Children Can Help Them Learn Numbers Faster. PLoS ONE, 2011. (To appear)
RAMSCAR, M.; YARLETT, D. Linguistic self-correction in the absence of feedback: A new approach to the logical problem of language acquisition. Cognitive Science, v. 31, n. 6, p. 927-960, 2007.
RAMSCAR, M.; YARLETT, D.; DYE, M.; DENNY, K.; THORPE, K. The effects of feature-label-order and their implications for symbolic learning. Cognitive Science, v. 34, n. 7, 2010. (In press).
SCHULTZ, W. Getting formal with dopamine and reward. Neuron, v. 36, n. 2, p. 241-263, 2002.
SEIDENBERG, M. S.; McCLELLAND, J. L. A distributed, developmental model of word recognition and naming. Psychological Review, v. 96, p. 523-568, 1989.
SIEGEL, S.; ALLAN, L. G. The widespread influence of the Rescorla-Wagner model. Psychonomic Bulletin & Review, v. 3, n. 3, p. 314-321, 1996.
SKOUSEN, R. Analogical modeling of language Dordrecht: Kluwer, 1989.
STROBL, C.; BOULESTEIX, A.-L.; KNEIB, T.; AUGUSTIN, T.; ZEILEIS, A. Conditional variable importance for random forests. BMC Bioinformatics, 9. 2008. Available at:<http://www.biomedcentral.com/1471-2105/9/307>
STROBL, C.; MALLEY, J.; TUTZ, G. An Introduction to Recursive Partitioning: Rationale, Application, and Characteristics of Classification and Regression Trees, Bagging, and Random Forests. Psychological Methods, v. 14, n. 4, p. 323-348, 2009.
TAGLIAMONTE, S.; BAAYEN, R. Models, forests and trees of York English: Was/were variation as a case study for statistical practice. 2010. Manuscript submitted for publication.
TREMBLAY, A.; BAAYEN, R. H. Holistic processing of regular four-word sequences: A behavioral and ERP study of the effects of structure, frequency, and probability on immediate free recall. In: WOOD, D. (Ed.). Perspectives on Formulaic Language Acquisition and Communication, 2010.
Van HEUVEN, W. J. B.; DIJKSTRA, A.; GRAINGER, J. Orthographic neighborhood effects in bilingual word recognition. Journal of Memory and Language, v. 39, p. 458-483, 1998.
VAPNIK, V. The nature of statistical learning theory Berlin: Springer-Verlag, 1995.
WAGNER, A.; RESCORLA, R. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: BLACK, A. H.; PROKASY, W. F. (Ed.). Classical conditioning ii. Appleton-Century-Crofts, 1972.
Downloads
Published
Issue
Section
License
Copyright (c) 2012 Revista Brasileira de Linguística Aplicada
This work is licensed under a Creative Commons Attribution 4.0 International License.
Autores de artigos publicados pela RBLA mantêm os direitos autorais de seus trabalhos, licenciando-os sob a licença Creative Commons BY Attribution 4.0, que permite que os artigos sejam reutilizados e distribuídos sem restrição, desde que o trabalho original seja corretamente citado.
