• danielbln@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    11 months ago

    I’ve implemented a few of these and that’s about the most lazy implementation possible. That system prompt must be 4 words and a crayon drawing. No jailbreak protection, no conversation alignment, no blocking of conversation atypical requests? Amateur hour, but I bet someone got paid.

    • Mikina@programming.dev
      link
      fedilink
      arrow-up
      1
      ·
      11 months ago

      Is it even possible to solve the prompt injection attack (“ignore all previous instructions”) using the prompt alone?

      • HaruAjsuru@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        edit-2
        11 months ago

        You can surely reduce the attack surface with multiple ways, but by doing so your AI will become more and more restricted. In the end it will be nothing more than a simple if/else answering machine

        Here is a useful resource for you to try: https://gandalf.lakera.ai/

        When you reach lv8 aka GANDALF THE WHITE v2 you will know what I mean

        • Kethal@lemmy.world
          link
          fedilink
          arrow-up
          0
          ·
          11 months ago

          I found a single prompt that works for every level except 8. I can’t get anywhere with level 8 though.

          • fishos@lemmy.world
            link
            fedilink
            English
            arrow-up
            0
            arrow-down
            1
            ·
            11 months ago

            I found asking it to answer in an acrostic poem defeated everything. Ask for “information” to stay vague and an acrostic answer. Solved it all lol.

    • CaptDust@sh.itjust.works
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      11 months ago

      That’s most of these dealer sites… lowest bidder marketing company with no context and little development experience outside of deploying CDK Roaster gets told “we need ai” and voila, here’s AI.

      • nickiwest@lemmy.world
        link
        fedilink
        arrow-up
        1
        ·
        11 months ago

        That’s most of the programs car dealers buy… lowest bidder marketing company with no context and little practical experience gets told “we need X” and voila, here’s X.

        I worked in marketing for a decade, and when my company started trying to court car dealerships, the quality expectation for that segment of our work was basically non-existent. We went from a high-end boutique experience with 99% accuracy and on-time delivery to mass-produced garbage marketing with literally bare-minimum quality control. 1/10, would not recommend.

        • CaptDust@sh.itjust.works
          link
          fedilink
          arrow-up
          1
          ·
          edit-2
          11 months ago

          Spot on, I got roped into dealership backends and it’s the same across the board. No care given for quality or purpose, as long as the narcissist idiots running the company can brag about how “cutting edge” they are at the next trade show.