d

  • mercano@lemmy.world
    link
    fedilink
    arrow-up
    14
    arrow-down
    1
    ·
    4 months ago

    AI doesn’t see a word as a sequence of letters, they just see it as a pointer to an entry in Words table.

      • Lambda@lemmy.ca
        link
        fedilink
        arrow-up
        12
        ·
        4 months ago

        Yeah, if words were actually encoded as 1-hot vectors this would be pretty trivial, but the rest of LLM training would be somewhere between infeasible and impossible. The actual embedding vectors obscure spelling even more.

        Side note: last time I checked, current embedding vectors were approximately 40 dimensional… Has that gone up significantly in the last couple of years?

    • chicken@lemmy.dbzer0.com
      link
      fedilink
      arrow-up
      3
      ·
      4 months ago

      Shouldn’t it help that it separated them out with underlines? How does this text break down in terms of tokens?

  • Juice@midwest.social
    link
    fedilink
    arrow-up
    10
    arrow-down
    1
    ·
    4 months ago

    I noticed this a few times I tried using ai to solve a wordle I was about to give up on. It isn’t just worthless, it like can not guess a 5 letter word based on a few rules like “second letter not a, d or g; first letter l, last letter w; word is not " lower”

    Its not that it can’t solve it, it can’t even guess slightly correct. I think ai language isn’t as connected to “spelling” as we think, I’ve heard of people using ai to translate instructions to Mandarin and then feeding the Mandarin query in because Mandarin is more “meaning dense” therefore uses less tokens and gets better answers

    • Abrinoxus@lemmy.todayOP
      link
      fedilink
      arrow-up
      4
      arrow-down
      1
      ·
      4 months ago

      yeah they do not think nor reason just most likely token prediction, most obv in small models but larger are a bit better at making you believe. emperors new clothes, i bet the house of cards will go down soon enough going to be interesting times at least

      • Juice@midwest.social
        link
        fedilink
        arrow-up
        1
        ·
        2 months ago

        Because its meaning dense? I guess it depends on the use case, might make for an interesting experiment

  • youcantreadthis@quokk.au
    link
    fedilink
    English
    arrow-up
    8
    ·
    4 months ago

    This technology is THEFUTURE if you don’t choke on its dick so hard it looks like you’re trying to fold your own erection between your ass cheeks you’re going to megahell if you don’t understand why we can’t go back to a time without it you’re just an insane Luddite moron

  • Rain World: Slugcat Game@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    transcript:
    [picture of claude chat:]
    claude: Wednesday, Monday, and Friday all contain the letter “D” — and so does Sunday and Saturday if you count the “d” at the end… wait, let me list them all carefully:

    • Mon_d_ay
    • Wednes_d_ay
    • Fri_d_ay
    • Sun_d_ay
    • Satur_d_ay

    So five days of the week contain the letter “d”: Monday, Wednesday, Friday, Sunday, and Saturday. Tuesday and Thursday are the only two that don’t.