Papers
Topics
Authors
Recent
Search
2000 character limit reached

Nearest Neighbor Representations of Neurons

Published 13 Feb 2024 in cs.CC, cs.DM, cs.LG, and cs.NE | (2402.08748v2)

Abstract: The Nearest Neighbor (NN) Representation is an emerging computational model that is inspired by the brain. We study the complexity of representing a neuron (threshold function) using the NN representations. It is known that two anchors (the points to which NN is computed) are sufficient for a NN representation of a threshold function, however, the resolution (the maximum number of bits required for the entries of an anchor) is $O(n\log{n})$. In this work, the trade-off between the number of anchors and the resolution of a NN representation of threshold functions is investigated. We prove that the well-known threshold functions EQUALITY, COMPARISON, and ODD-MAX-BIT, which require 2 or 3 anchors and resolution of $O(n)$, can be represented by polynomially large number of anchors in $n$ and $O(\log{n})$ resolution. We conjecture that for all threshold functions, there are NN representations with polynomially large size and logarithmic resolution in $n$.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (17)
  1. Noga Alon and Văn H Vũ “Anti-Hadamard matrices, coin weighing, threshold gates, and indecomposable hypergraphs” In Journal of Combinatorial Theory, Series A 79.1 Elsevier, 1997, pp. 133–160
  2. “On the complexity of depth-2 circuits with threshold gates” In International Symposium on Mathematical Foundations of Computer Science, 2005, pp. 107–118 Springer
  3. “Weights of exact threshold functions” In International Symposium on Mathematical Foundations of Computer Science, 2010, pp. 66–77 Springer
  4. Vasken Bohossian, Marc D Riedel and Jehoshua Bruck “Trading Weight Size for Circuit Depth: A Circuit for Comparison” California Institute of Technology, 1998 URL: https://authors.library.caltech.edu/26049/1/etr028.pdf
  5. “Simulating threshold circuits by majority circuits” In Proceedings of the twenty-fifth annual ACM symposium on Theory of computing, 1993, pp. 551–560
  6. Péter Hajnal, Zhihao Liu and György Turán “Nearest Neighbor Representations of Boolean Functions” In Information and Computation 285 Elsevier, 2022, pp. 104879
  7. Kristoffer Arnsfelt Hansen and Vladimir V Podolskii “Exact threshold circuits” In 2010 IEEE 25th Annual Conference on Computational Complexity, 2010, pp. 270–279 IEEE
  8. Johan Håstad “On the size of weights for threshold gates” In SIAM Journal on Discrete Mathematics 7.3 SIAM, 1994, pp. 484–492
  9. Alan L Hodgkin and Andrew F Huxley “A quantitative description of membrane current and its application to conduction and excitation in nerve” In The Journal of physiology 117.4 Wiley-Blackwell, 1952, pp. 500
  10. Thomas Hofmeister “A note on the simulation of exponential threshold weights” In International Computing and Combinatorics Conference, 1996, pp. 136–141 Springer
  11. Kordag Mehmet Kilic and Jehoshua Bruck “Neural Network Computations with DOMINATION Functions” In 2021 IEEE International Symposium on Information Theory (ISIT), 2021, pp. 1029–1034 IEEE DOI: 10.1109/ISIT45174.2021.9517872
  12. Kordag Mehmet Kilic, Jin Sima and Jehoshua Bruck “On algebraic constructions of neural networks with small weights” In 2022 IEEE International Symposium on Information Theory (ISIT), 2022, pp. 3007–3012 IEEE DOI: 10.1109/ISIT50566.2022.9834401
  13. Kordag Mehmet Kilic, Jin Sima and Jehoshua Bruck “On the Information Capacity of Nearest Neighbor Representations” In 2023 IEEE International Symposium on Information Theory (ISIT), 2023, pp. 1663–1668 DOI: 10.1109/ISIT54713.2023.10206832
  14. Warren S McCulloch and Walter Pitts “A logical calculus of the ideas immanent in nervous activity” In The bulletin of mathematical biophysics 5 Springer, 1943, pp. 115–133
  15. Marvin Minsky and Seymour A Papert “Perceptrons, reissue of the 1988 expanded edition with a new foreword by Léon Bottou: an introduction to computational geometry” MIT press, 2017
  16. Saburo Muroga “Threshold logic and its applications”, 1971
  17. Frank Rosenblatt “The perceptron: a probabilistic model for information storage and organization in the brain.” In Psychological review 65.6 American Psychological Association, 1958, pp. 386
Citations (2)

Summary

No one has generated a summary of this paper yet.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.